AD-A078  632  BOEING  COMPUTER  SERVICES  INC  SEATTLE  WASHINGTON  SYSTEMS  ENG  I — ETC 
MANAGEMENT  TOOLS  CASE  STUDY,  (U) 

SEP  79  J  R  BROWN,  L  S  HAMMOND  F30602-78-C-0044 

limn  A«;STPTFT> _ P  ATir-TR-7C)-777 _ N 


I  OF  2 


MICROCOPY  RESOLUTION  TEST  CHART 


ADAO 


RADC-TR-79-223 

Final  Technical  Report 
Soptombor  1979 


CO 

1C  < 
00 


MANAGEMENT  TOOLS  CASE  STUDY 


Boeing  Computer  Services  Company 

John  R.  Brown 
Linda  S.  Hammond 


* 


APPROVED  FOR  PUBLIC  RELEASE;  DISTRIBUTION  UNLIMITED 


ROME  AIR  DEVELOPMENT  CENTER 

Air  Force  Systems  Command 

Griff iss  Air  Force  Base,  New  York  13441 


This  report  has  been  reviewed  by  the  RADC  Information  Office  (01)  and 
is  releasable  to  the  National  Technical  Information  Service  (NTIS) .  At  NTIS 
it  will  be  releasable  to  the  general  public,  including  foreign  nations. 

RADC-TR-79-223  has  been  reviewed  and  is  approved  for  publication. 


APPROVED:  7 e  ^  ^ 

ROGER  W.  WEBER 
Project  Engineer 


WENDALL  C.  BAUMAN,  Colonel,  USAF 
Chief,  Information  Sciences  Division 


Acting  Chief,  Plans  Office 


If  your  address  has  changed  or  if  you  wish  to  be  removed  from  the  RADC 
mailing  list,  or  if  the  addressee  is  no  longer  employed  by  your  organization, 
please  notify  RADC  (ISIE),  Griffiss  AFB  NY  13441.  This  will  assist  us  in 
maintaining  a  current  mailing  list. 

Do  not  return  this  copy.  Retain  or  destroy. 


SECURITY  CLASSIFICATION  of  This  PAGE  (When  Data  Entered ) 

(]|)REP0RT  DOCUMENTATION  page  I  BEFORE  COMPl.hflNO  FORM 

1  ppp^rr..MliuagB  2.  GOVT  ACCESSION  NO.  3  RECIPIENT'S  CAT  ALOG  NUMBER 

RADc/rR-79-223  _ 

■*r~TtTgrnna  iu ~Fa  \  5  TVPe  of  report  a  PEBiooxovEf 


MANAGEMENT  TOOLS  CASE  STUDY  t 


j  author  (•) 

John  R. /Brown 
Linda  S. ^Hammond 

Y  PERFORMING  ORGANIZATION  NAME  AND  ADDRESS'^.  , 

Boeing  Computer  Services  Ctrtipjliy'^  i  ■  • 
Systems  Engineering  Technology  Federal  S 
P'  0  Box  24  346 

Seattle  WA  98124 _ 

1  1.  CONTROLLING  OFFICE  NAME  ANO  ADDRESS 

Rome  Air  Development  Center  (ISIE) 
Griffiss  AFB  NY  13441 


A  5  type  of  report  a  pejjioc  covered 

I J  J^inal  TiEchnical  XeP®f# 

•n  'Bee  77 — 'Dec  78^ _ _ _ 

/  «&PERroR«ll/AoR<i  REPORT  NUMBER 

_  N/A  _ j _ 

B.  CONTRACT  OR  GRANT  NUMBER'S) 


\\j  ,  F3|)6^2-78-C-0p44/Z< 


Systems  Gp 


1C  PROGRAM  ELEMENT.  PROJECT  T  ASK 
AREA  A  WORK  UNIT  NUMBERS 


62Z02  F  (  /  7  )  X  ?  ! 

5581jl804  - ■*- 

ET<SeP0R7  DATE 


MOSIT  JRING  AGENC  1  NAME  §  ADDRESS.'//  different  from  Controlling  Ot(ice)  J  IS  SECURITY  CLASS,  (of  this  report) 


\i)X 


JJNCLASSIFIED _ 

15  a.  DECLASSIFICATION  DOWNGRADING 
.  SCHEDULE 

N/A 


M6  DIST  RIBUTION  ST  ATEMC  N  T  (ol  thi  a  Report) 


Approved  for  public  release;  distribution  unlimited. 


ftT.  DIS1  Rl  BUT  ION  ST  ATEMENT  (ol  the  abstract  entered  In  Block  20,  II  different  from  Report) 


'  18  SUPPL  FMENTARY  NOTES 

!  RADC  Project  Engineer:  Roger  W.  Weber  (TSIE) 


i 

. 

M9  KEY  W' 


WORDS  (Continue  on  reverse  aide  il  necessary  and  identify  by  block  number) 


Computer  Software 
Program  Management 
Management  Engineering 


Management  Practices 
Software  Engineering 


20  ABSTRACT  (Continue  on  reverse  side  If  necessary  and  Identify  by  block  number i 

TiThis  study  investigated  documentation  and  procedures  for  the  NASA  Integrated 
Programs  for  Aerospace  Vehicle  Design  (IPAD)  Program.  The  purpose  was  to  assess 
IPAD's  value  to  a  proposed  R&D  program  for  evaluating  the  application  of 
management  principles  and  disciplines  to  the  management  of  software  development. 
To  determine  the  actual  relationship  between  IPAD  objectives  and  the  application 
of  management  techniques,  a  survey  was  given  to  key  IPAD  management  and 
technical  personnel.  A  summary  of  the  responses  is  presented  in  Appendix  C. 


DD  ,  j  an  *73  1473 


,  UNCLASSIFIED 

J  SECURITY  CL  AS5IFIC  ATlON  OF  THIS  PAGE  rWi»n  D«l»  Enf«r»d) 

Yl  ISfca' 


MM 


UNCLASSIFIED 


SECURITY  CLASSIFICATION  OF  THIS  PAGEfHTien  D«I«  Enleril) 


""The  report  discusses:  key  responsibilities;  formal  and  infoi  lal 
direction;  status,  progress,  and  performance  indicators.  A  generalized 
management  tool  model  is  presented  to  summarize  the  studied  relationships 
between  status  indicators,  program  objectives,  management  techniques  (MT) , 
MT  indicators,  and  key  responsibilities.  Hypothetical  IPAD  Situation 
Scenarios  were  generated  to  illustrate  the  steps  taken  in  the  IPAD 
environment  to  implement  the  management  practices  and  tools  studied. 

The  report  contains  sample  reports  and  forms  as  well  as  checklists. 


i-fr 


UNCLASSIFIED 


SECURITY  CLASSIFICATION  of  this  PAOEflFh*n  Dm tm  Enfrmd) 


.  ... 


CONTENTS  i 

Page 

1.0  INTRODUCTION  AND  SUMMARY  1 

1.1  BACKGROUND/OBJECTIVE  1 

1.2  PROJECT  APPROACH/SCOPE  2 

1.2.1  Task  1  3 

1.2.2  Task  2  3 

1.2.3  Task  3  3 

1.2.4  Task  4  3 

1.2.5  Task  5  4 

1.3  OVERVIEW  4 

2.0  IPAD  ENVIRONMENT  5 

2.1  IPAD  DESCRIPTION  5 

2.2  IPAD  MANAGEMENT  TECHNIQUES  5 

2.2.1  Project  Organization  and  Administration  6 

2.2.2  Project  Planning  11 

2.2.3  Project  Evaluation  and  Control  13 

2.3  IPAD  DEVELOPMENT  OBJECTIVES  15 

2.3.1  Management  Life  Cycle  Objectives  16 

2.3.2  User  Interface  Objectives  16 

2.3.3  Technical  Excellence  Objectives  17 

3.0  SURVEY  DESCRIPTION  19 

3.1  SURVEY  OBJECTIVES  19 

3.2  SURVEY  TECHNIQUE  19 

3.3  SURVEY  RESULTS  20 


i-Q 


CONTENTS  (continued) 


4.0  SURVEY  RESULTS 

4.1  SURVEY  -  PART  1 

4.2  SURVEY  -  PART  2 

4.3  SURVEY  -  PART  3 


4.3.1  Modeling 

4.3.2  Example 

4.3.3  Summary  Survey  Results 

5.0  INDIVIDUAL  RESPONSIBILITIES  AND  DIRECTION 

5.1  KEY  RESPONSIBILITIES 

5.1.1  Business  Manager 

5.1.2  ITAB  Interface  Manager 

5.1.3  Engineering  Development  Manager 

5.1.4  Software  Development  Manager 

5.1.5  Technical  Director/Chief  Designer 

5.1.6  IPAD  Systems  Integrator/ Assistant  Chief  Designer 

5.1.7  Technical  Leader  —  Methodology,  Standards  and 
Tools 

5.1.8  Program  Support  Manager 

5.1.9  IPAD  System  Technical  Leaders —  Major  Components 

5.2  FORMAL  AND  INFORMAL  DIRECTION 

5.3  RELATION  BETWEEN  DIRECTION  AND  PROGRAM  OBJECTIVES 
6.0  STATUS,  PROGRESS  AND  PERFORMANCE  INDICATORS 

6.1  GENERAL  INFORMATION 

6.2  MONITORING  STATUS,  PROGRESS  AND  PERFORMANCE 

6.3  IPAD  INDICATORS 

6.4  INDICATOR  RESPONSIBILITY 

6.5  INDICATOR/OBJECTIVE  RELATIONSHIP 

6.6  MANAGEMENT  TECHNIQUE  INDICATORS 

6.7  ADDITIONAL  MANAGEMENT  TECHNIQUE  INDICATORS 

7.0  GENERALIZED  MANAGEMENT  TOOL  MODEL 

7.1  SELECTION  OF  TECHNIQUES  AND  INDICATORS 

7.2  SUMMARY 


•Hi  JfiAWWl «  >  W»*->  * 


CONTENTS  (continued) 


APPENDIX  A 
APPENDIX  B 
APPENDIX  C 
APPENDIX  D 

APPENDIX  E 

APPENDIX  F 
APPENDIX  G 


SUMMARY  OF  RESPONSES  TO  PART  1  OF  SURVEY 
SUMMARY  OF  RESPONSES  TO  PART  2  OF  SURVEY 
SUMMARY  OF  RESPONSES  TO  PART  3  OF  SURVEY 
SUMMARY  OF  ALGORITHM  INDICES  FOR  PART  3 
OF  SURVEY 

SUMMARY  OF  STRONG  AND  MODERATE  RESPONSES 
TO  PART  3  OF  SURVEY 

HYPOTHETICAL  IPAD  SITUATION  SCENARIOS 
IPAD  SCENARIO  RESOLUTION 


Page 

110 

111 

112 

113 

1 14 

115 
117 


iii 


LIST  OF  FIGURES 

Page 

4. 3. 2- 1  EXAMPLE  SURVEY  RESPONSES  26 

4. 3.2- 2  INDICES  OF  OBJECTIVE/TECHNIQUE  RELATIONSHIP  26 

4. 3. 3- 1  STRONG  OBJECTIVE/TECHNIQUE  RELATIONSHIPS  29 

5.2- 1  FORMAL  PROGRAM  ORGANIZATION  AND  40 

REPORTING  RELATIONSHIPS 

5.3- 1  RESPONSIBILITIES  FOR  SCENARIO  1  49 

5.3- 2  RESPONSIBILITIES  FOR  SCENARIO  2  51 

6.2- 1  WBS  AS  BASIS  OF  IPAD  PLANNING  AND  CONTROL  55 

6.2- 2  SAMPLE  IPAD  WORK  AUTHORIZATION  FORM  56 

6.3- 1  SAMPLE  SOFTWARE  NOTEBOOK  COVER  SHEET  59 

6.3- 2  SAMPLE  WALKTHROUGH  APPROVAL  REPORT  61 

6.3- 3  SAMPLE  QUALITY  ASSURANCE  CHECK  SHEET  62 

6.3- 4  SAMPLE  IDAP  FULL  PDL  REPORT  66 

6.3- 5  SAMPLE  DOCUMENTATION  PROCESS  CHECK  SHEET  69 

6.3- 6  SAMPLE  SOFTWARE  PROBLEM  REPORT  72 

6.3- 7  SAMPLE  REQUIREMENTS  ANALYSIS  FORM  74 

6.3- 8  SAMPLE  REQUIREMENTS  ANALYSIS  CRITERIA  CHECKLIST  75 

6. 3- 9(1)  SAMPLE  533-M  REPORT  (WBS  LEVEL  2)  77 

6 .3- 9(2)  SAMPLE  533-M  REPORT  (WBS  LEVEL  I)  78 

6. 3- 9(3)  SAMPLE  533-M  REPORT  (TOTAL  PROGRAM)  79 

6.3- 10  SAMPLE  INTEGRATED  RESOURCE/SCHEDULE  REPORT  81 

6.3- 11  SAMPLE  WORK  SCHEDULE  STATUS  REPORT  82 


IV 


LIST  OF  FIGURES  (continued) 


Pag® 


6.4- 1  RELATIONSHIP  OF  FORMALLY-DIRECTED  RESPONSIBILITIES  86 

TO  INDICATORS 

6.5- 1  RELATIONSHIP  BETWEEN  INDICATORS  AND  OBJECTIVES  89 

7.1- 1  GENERALIZED  MANAGEMENT  TOOL  MODEL  100 

7.1- 2  GENERALIZED  MODEL  OF  SCENARIO  A  104 

7.1- 3  GENERALIZED  MODEL  OF  SCENARIO  B  108 


v 


4 


EVALUATION 

The  software  industry  has  experienced  serious  problems 
with  cost  overruns,  late  deliveries,  poor  reliability  and 
customer  dissatisfaction.  It  has  been  claimed  that  there 
are  no  disciplined  ways  to  determine  the  qualifications  of 
available  resources;  no  uniformly  agreed  upon  methods  of 
assessing  performance;  no  reliable  ways  to  predict  the 
ultimate  costs  and  schedule  or  the  impact  of  changed  or 
external  events. 

It  is  evident  that  other  industries  are  less  susceptible 
to  these  problems,  primarily  because  there  has  been  a 
deliberate  attempt  to  apply  basic  engineering  principles 
in  a  disciplined  fashion. 

RADC  TPO  R5A ,  Software  Cost  Reduction  in  one  of  its 
thrusts,  addresses  the  managerial  aspects  of  Software 
Definition,  Design,  Construction,  and  Verification.  In  this 
thrust  the  goal  is  to  characterize  or  model  software  in 
terms  of  its  three  distinct  basic  processes:  Product  ion  of 
the  items  to  be  delivered;  Assessment  of  that  production  for 
status,  performance,  and  progress  indications;  and  P lanning 
(and  re-planning)  of  production  activities  based  upon 
assessment  data. 
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One  purpose  for  generating  process  descriptions 
is  to  provide  a  framework  for  guiding  future  R&D  activities 
related  to  software  development  and  production  management* 
These  related  activities  were  seen  as  consisting  of  three 
types:  (1)  definition  of  software  engineering  policies, 

methods  and  standards;  (2)  assessment  of  tools,  existing 
and  potentially  new,  in  terms  of  benefits  to  be  derived 
through  their  deliberate  incorporation  into  the  software 
production  processes;  and  (3)  preliminary  design  of  an 
advanced  Software  Cost /Schedule  Tracking  System. 

Before  embarking  on  such  a  modeling  effort,  it  was 
deemed  desirable  to  identify  a  large  on-going  development 
program  for  which  a  concerted  effort  was  made  to  select 
and  employ  modern  software  engineering  practices  to  the 
maximum  extent  practical.  We  felt  that  such  a  real  world 
referent  would  be  an  excellent  basis  for  generating  and 
validating  the  desired  models.  One  such  program,  NASA's 
Integrated  Programs  for  Aerospace  Vehicle  Design  (IPAD), 
was  identified  as  a  candidate. 

This  contractual  effort.  Management  Tools  Case  Study, 
was  undertaken  to  determine  whether  the  IPAD  Program  would 
be  a  satisfactory  environment  within  which  a  thorough  and 
conclusive  evaluation  of  modern  software  management  tools 
and  methods  could  be  accomplished.  During  the  course  of 
the  contract,  five  tasks  were  accomplished  via  review  of 
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documentation  and  reports,  interviews  with  management 
and  t  echn ical  individuals,  and  through  completion  of  a 
comprehensive  questionnaire.  Research  indicated  that  at 
least  19  indicators  of  project  status,  progress  and 
performance  are  available  on  IPAD.  The  relationships 
and  dependencies  between  program  objectives,  management 
techniques,  indicators  of  status,  progress  and 
performance,  program  responsibilities,  and  indicators 
of  technique  effectiveness  were  incorporated  into  a 
generalized  management  tool  model.  This  Final  Report 
describes  the  model,  summarizes  the  survey  results, 
details  the  techniques  used  on  IPAD,  and  through 
hypothetical  scenarios,  illustrates  the  relationship 
between  communicated  direction  and  identified  development 
obj  ect ives . 

fPjOUUy.  Co  . 

ROGE(u  W.  WEBER 
Project  Engineer 
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1.0  INTRODUCTION  AND  SUMMARY 

1.1  BACKGROUND/OBJECTIVE 

The  software  industry  has  experienced  serious  problems  with  cost  overruns,  late 
deliveries,  poor  reliability  and  customer  dissatisfaction.  A  common  rationale  for 
these  problems,  voiced  both  by  the  developers  and  the  buyers  of  computing 
software,  is  that  software  production  is  a  set  of  essentially  unmanageable  activi¬ 
ties.  It  is  claimed  that  there  are  no  disciplined  ways  to  determine  the  qualifica¬ 
tions  of  available  resources;  no  uniformly  agreed  upon  methods  of  assessing  per¬ 
formance;  no  reliable  ways  to  predict  the  ultimate  costs  and  schedule  or  the 
impact  of  changes  or  external  events. 

It  is  evident  that  other  industries  are  less  susceptible  to  these  problems,  primarily 
because  there  has  been  a  deliberate  attempt  to  apply  basic  engineering  principles 
in  a  disciplined  fashion.  While  the  software  industry  seems  enamored  of  the  term 
"software  engineering",  disciplined  application  of  engineering  principles  to  the 
software  production  process  has  been  rare. 

For  example,  one  basic  engineering  principle  is  that  activities  comprising  a  process 
should  be  "staged"  in  an  orderly  fashion,  to  guarantee  that  products  from  one 
activity  are  available  in  a  timely  fashion  for  subsequent  activities.  In  software 
production,  however,  this  principle  is  not  generally  implemented.  The  documen¬ 
tation  of  requirements,  designs  and  plans  for  the  software  system  are  activities 
which  are  frequently  postponed  (if  they  are  performed  at  all),  even  though  this 


1 


documentation  is  the  basic  framework  upon  which  the  software  itself  should  be 
built. 

Engineering,  As  it  is  applied  to  other  industries,  is  concerned  with  three  distinct 
processes:  Production  of  the  items  to  be  delivered;  assessment  of  that  production 
for  status,  performance,  and  progress  indications;  and  planning  (and  re-planning)  of 
the  production  activities,  based  upon  the  data  the  assessment  process  provides.  The 
production  process  concerns  itself  principally  with  technical  activities;  the 
assessment  and  planning/re-planning  processes  are  complementary  managerial 
activities.  BCS  believes  that  it  is  possible  to  apply  the  same  engineering  or 
management  principles  long  used  in  other  industries,  such  as  bridge  building  and 
airplane  manufacturing,  to  software  production;  i.e.,  that  software  development  is 
essentially  a  fabrication  process  subject  to  proven  techniques  for  monitoring  and 
planning. 

In  seeking  to  confirm  the  above-stated  belief,  Boeing  Computer  Services  (BCS) 
Company  is  conducting  a  Rome  Air  Development  Center  (R  ADC)  funded 
investigation  of  the  documentation  and  procedures  for  ine  NASA  Integrated 
Programs  for  Aerospace-Vehicle  Design  (IPAD)  Program.  The  investigation  will 
assess  IPAD's  value  to  a  proposed  Research  and  Development  Program  for 
evaluating  the  application  of  management  techniques  (principles,  disciplines)  to  the 
management  of  software  development. 

1.2  PROJECT  APPROACH/SCOPE 

In  order  to  determine  the  suitability  of  IPAD  as  a  vehicle  for  evaluating  the  use  of 
management  techniques  in  software  development,  five  independent  but  strongly 
related  tasks  will  be  performed. 
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1.2.1  Task  1 


Examine  IPAD  documentation  and  identify  management  techniques  being  applied  to 
the  project.  Documentation  concerning  the  following  items  was  investigated: 

•  Work  Breakdown  Schedule 

•  Milestone  Schedule 

•  Review  Schedule,  Plans,  Objectives,  Participants 

•  Detailed  Task  Assignments 

•  Procedures,  Forms,  Tools 

•  Organization  Structure 

1.2.2  Task  2 

Examine  IPAD  documentation  relative  to  the  customer's  program  objectives,  the 
contractual  agreement,  and  the  statement  of  work  to  determine  the  development 
objectives  of  IPAD.  Elicit  the  cooperation  of  IPAD  management  to  determine  the 
relationship  between  development  objectives  and  management  techniques  actually 
being  applied. 

1.2.3  Task  3 

Identify  IPAD  individuals  with  key  functional  responsibilities  and  determine  what 
formal  and  informal  direction  these  individuals  have  received.  Relate  the  formal 
and  informal  direction  upon  which  these  individuals  rely  to  discharge  their 
responsibilities  to  the  development  objectives  identified  in  Task  2. 

1.2.4  Task  4 

Identify  specific  indicators  used  by  IPAD  to  assess  status,  progress  and  performance. 
These  indicators  will  be  determined  from  the  management  plan  and  the  individuals 
with  key  functional  responsibilities  identified  in  Task  3.  Relate  these  indicators  to 
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the  development  objectives  identified  (in  Task  2)  and  the  direction  communicated  (in 
Task  3). 

1.2.5  Task  5* 

The  indicators  which  IPAD  is  using  to  assess  status,  progress  and  performance  shall 
be  related  to  the  Management  disciplines  or  techniques  being  applied.  An  analysis 
will  be  made  of  the  overall  relationship  between  individuals  with  key  responsibilities 
viewing  indicators  of  status  which  give  insight  into  techniques  used  to  manage  the 
project  and  achieve  the  program  objectives. 

1.3  OVERVIEW 

The  interim  report  covered  work  performed  under  Tasks  1  and  2,  identified  in  the 
previous  section.  Candidate  management  techniques  and  development  objectives 
were  identified.  In  order  to  determine  the  relationship  between  the  techniques  and 
objectives,  a  survey/questionnaire  was  prepared  and  conducted  using  key  IPAD 
management  and  technical  personnel.  The  survey  results  are  presented  in  Section  4 
of  this  document. 

Tasks  3,  4  and  5  were  researched  and  documented  from  May  to  December,  1978. 
Key  individuals  were  identified  and  interviewed  and  documentation  was  studied.  A 
generalized  management  tool  model  was  developed  to  illustrate  the  relationship 
between  techniques,  objectives,  indicators  of  status,  progress  and  performance, 
indicators  of  technique  effectiveness  and  individuals  with  key  responsibilities.  The 
model  is  presented  in  Section  7. 


*Task  5  added  via  an  Engineering  Change  Proposal  on  September  13,  1978. 
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2.0  IPAD  ENVIRONMENT 

2.1  IPAD  DESCRIPTION 

The  IPAD  system  is  a  general  purpose  interactive  computing  system  developed  to 
support  engineering  design  processes.  Its  primary  function  is  to  handle  engineering 
data  and  management  data  associated  with  the  design  process.  It  is  intended  to 
support  continuous  design  activities  of  a  typical  company  mix  of  multiple  develop¬ 
ment  projects.  IPAD  serves  management  and  engineering  staffs  at  all  levels  of 
design  (conceptual,  preliminary,  and  final)  and  aids  in  the  assembly  and  organi¬ 
zation  of  design  data  for  manufacturing  processes. 

The  IPAD  system  design  will  support  generation,  storage  and  management  of  large 
quantities  of  data.  Its  capacity  will  only  be  limited  by  the  hardware  configurations 
selected  by  each  company.  The  system  is  intended  for  use  in  a  distributed 
computing  environment  having  one  or  more  central  host  computing  systems  and 
many  remote  computing  systems.  The  number  of  terminals  supported  may  range  to 
several  hundred  and  may  be  distributed  across  the  host  and  remote  systems.  The 
IPAD  software  will  function  on  the  "third  generation"  computer  complexes  in  use 
today  by  large  aerospace  corporations. 

2.2  IPAD  MANAGEMENT  TECHNIQUES 

Examination  of  IPAD  documentation,  in  conjunction  with  the  BCS  Systematic 
Software  Development  and  Maintenance  (SSDM)  Project  Management  Guide,  has 
identified  many  candidate  management  techniques.  These  techniques  can  be 
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grouped  in  three  broad  categories  which  generally  depict  their  function  within  a 
software  development  project.  These  categories  are:  1)  Project  organization  and 
administration,  2)  Project  planning  and  3)  Project  evaluation  and  control. 


Project  organization  and  administration  includes  management  techniques  used  to 
organize  the  technical  staff,  assemble  the  project's  resources,  establish  a  method 
for  assigning  and  delegating  work  responsibility  and  project  communication,  and 
evaluate  the  quality  of  the  project's  technical  work. 

Project  planning  includes  the  essential  elements  of  the  project  plan,  a  mechanical 
means  to  schedule  a  project's  work  tasks  through  the  systematic  allocation  of 
project  resources,  and  established  standards  of  performance. 

Project  evaluation  and  control  includes  functions  which  assess  the  project's  overall 
progress.  Methods  for  analyzing  fiscal  reports,  techniques  for  conducting  project 
reviews,  and  approaches  toward  corrective  action  and  change  control  are  examples 
of  project  evaluation  and  control  techniques. 

2.2.1  Project  Organization  and  Administration 

2.2.1. 1  Project  Manager  Concept 

The  project  manager  has  total  responsibility  and  authority  to  lead  a  team  of 
qualified,  skilled  individuals  to  accomplish  the  goals  of  the  project.  He  has  to 
maintain  technical  and  resource  control  at  all  times.  He  will  separate  and  delegate 
the  project  management  responsibilities  and  authorities  to  other  key  members  of  the 
team,  thus  allowing  him  to  focus  his  attention  on  general,  long-term  project 
direction,  financial/schedule  control  and  external  interfaces.  An  assistant  project 
manager  may  be  appointed  to  conduct  overall  day-to-day  operations. 
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2.2.1. 2  Work  Breakdown  Schedule/Structure  (WBS) 


Describes  the  subdividing  of  project  effort.  The  total  project  work  is  divided  into 
major  groups,  the  groups  subdivided  into  tasks,  and  so  on.  The  lowest  level  of 
subdivision  is  small  enough  to  permit  control  and  visibility  without  excessive 
administrative  "red  tape".  Work  performed  in  major  elements  is  the  sum  of  all  the 
work  specified  in  the  elements  below  it. 

The  work  breakdown  is  sometimes  accompanied  by  a  schedule  of  task  milestones  in  a 
graphical  and/or  tabular  form.  The  graphical  WBS  can  also  incorporate  a  colored 
horizontal  bar  which  indicates  status  and  progress  in  relation  to  plan. 

2.2. 1.3  Team  Organization 

The  organization  of  a  team  to  depart  from  the  normal  "programmer  type"  organi¬ 
zation.  The  project  manager  determines  the  mix  that  accommodates  personality 
differences  while  allowing  for  satisfaction  of  user  needs. 

2.2. 1.3.1  Egoless  Teams 

The  team  designs  the  code,  individuals  produce  it,  and  all  members  review  it.  The 
team  gives  assurance  that  each  functional  module  is  understood  and  complete. 

2.2. 1.3. 2  Chief  Programmer  Teams 

A  group  of  specialists  organized  to  support  top-down  development.  A  Chief 
Programmer  designs  and  produces  all  the  code  for  the  critical  sections,  defines 
programmer  support,  reviews  design  and  code  produced  by  others  and  controls  its 
integration  into  the  total  system. 


2.2. 1.4  Programmer's  Handbook 


A  loose-leaf  binder  containing  written  information  and  communications  needed  by 
programmers.  The  handbook  contains: 

a.  Description  of  technical  requirements 

b.  Baseline  design 

c.  Support  software 

d.  Test  procedures 

e.  Hardware  resources  available 

f.  Summary  of  documentation  plan 

g.  Programming  and  documentation  standards 

2.2. 1.5  Project  File 

A  set  of  management  data  which  provides  for  centralization  of  information  about 
the  project  and  its  activities.  Contains: 

a.  Reports,  information  for  accounting  purposes 

b.  Project  plan 

c.  Minutes  of  meetings  and  reviews 

d.  Correspondence 

e.  Documentation  of  findings,  conclusions,  recommendations 

f.  Worksheets  and  other  analyses 

g.  Change  Requests 

h.  Project  Performance  Data 

i.  Problem  Reports 

j.  Lessons  learned 

k.  Location  of  major  project  documentation 


2.2. 1.6  Software  Notebook 


A  summary  notebook  produced  for  each  functional  module  of  project  work.  It 
focuses  attention  on  process,  component  and  module  documentation,  thereby 
enforcing  the  idea  that  documentation  should  evolve  throughout  project  develop¬ 
ment.  It  is  a  collection  of  development  and  test  information  and  supports  detailed 
planning  of  project  effort  and  status.  It  is  used  to  accomplish  planning  and  control 
of  components,  units  and  modules.  The  first  page  of  the  notebook  is  generally  a 
summary  sheet  containing  a  list  of  the  many  tasks  which  are  necessary  to  complete 
the  module.  For  example,  it  is  likely  to  contain  due  dates,  completion  dates,  as  well 
as  the  names  of  the  originator  and  reviewer. 

2. 2. 1.7  Support  Library 

Tool  used  for  storing  and  recording  computer  program  development  data.  It  includes 
all  computer  and  manual  procedures  needed  to  manipulate  library  data,  guaranteeing 
that  once  a  module  has  been  placed  in  a  controlled  library,  changes  are  not  made 
without  author  knowledge.  Its  design  permits  isolation  and  delegation  of  record 
keeping  and  clerical  operations.  It  includes: 

-  source  code  (in  human  readable  form)  of  official  copy 

-  object  code  (in  machine  readable  form)  of  official  copy 

-  control  data 

As  the  development  effort  continues,  test  data,  source  and  object  code  and 
information  to  support  the  management  of  the  outdated  or  revised  versions  of  a 
module  are  added  to  the  library. 

2.2. 1.8  Personnel  Selection  Requirements  Form 

A  form  used  as  a  tool  for  personnel  selection.  It  aids  in  the  analysis  of  individual 
requirements  and  development  of  a  staff  training  plan.  Skill  requirements  are  used 
as  the  criteria  for  identifying  and  selecting  project  members. 
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2.2. 1.9  Work  Authorization  Form 

A  formal  means  for  delegating  work  in  the  project  environment  which  makes  the 
work  itself  accountable.  It  is  a  tool  to  be  used  when  planning  and  scheduling  project 
activity.  As  a  minimum,  it  includes  specific  budgets,  schedules  and  descriptions  of 
the  work  to  be  done. 

2.2.1.10  Project  Control  Room 

A  tool  for  promoting  good  project  communications.  A  room  assigned  to  the  project 
for  their  exclusive  use  which  is  suitable  for  review  meetings,  working  sessions,  and 
display  of  project  data. 

Usually  a  project  room  will  contain  displays  of  detailed  schedules,  budget  cost 
charts,  delinquent  items,  action  item  assignments,  project  technical  performance, 
status  assessment,  critical  path  events,  test  results,  etc. 

2.2.1.11  Formal  Inspection  of  Design  Documentation  and  Code 

Formal  way  of  independently  reviewing  the  quality  of  project  design,  documentation 
and  code.  Techniques  included  in  this  category  are  walk-throughs,  inspections,  and 
QA  reviews  and  audits. 

2.2.1.11.1  Walk-through 

A  series  of  reviews,  each  with  different  objectives  and  occurring  at  different  times 
during  the  development  process.  Typically,  it  is  a  presentation  of  a  module's 
metacode  or  control  graph,  where  the  author/programmer  "walks"  the  other  team 
members  through  the  module  logic  and  data  flow.  Team  members  evaluate  program 
logic  to  detect  errors  and  assure  thorough  testing  of  each  module.  A  report 
documenting  the  results  of  the  walk-through  is  usually  produced. 
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2.2.1.11.2  Inspections 

A  more  formalized  review  which  requires  more  planning,  has  more  visibility  and 
assumes  more  management  involvement  than  a  walk-through.  Provides  a  way  to 
capture  statistics  on  the  types,  the  numbers  and  the  places  where  errors  occur. 
These  statistics  act  as  feedback  to  the  development  team  and  project  management. 

2.2.1.11.3  Quality  Assurance  Reviews  and  Audits 

Independent  QA  Reviews  conducted  at  critical  points  during  software  development 
to  ensure  requirements  satisfaction,  completeness  of  design  specification,  verifi¬ 
cation  of  module  interfaces,  and  adequacy  of  development  and  independent  testing. 

2.2.2  Project  Planning 

2.2.2. 1  Resource  Allocation  Sheets 

Provide  a  formal  way  for  distributing  the  project  staff  to  elements  of  the  WBS. 
Establishes  activity  start  and  completion  dates,  determines  schedule  for  facilities, 
clerical  needs,  machine  times  and  user  involvement.  It  is  essentially  a  bar  chart 
showing  calendar  time  horizontally  and  project  resources  vertically.  Normally  all 
Resource  Activity  Sheets  are  summarized  on  Master  Activity  Sheets  which  are  a 
selected  listing  of  WBS  items. 

2.2.2. 2  Resource  Requirements  Summary 

Denotes  the  resources  necessary  to  execute  the  project.  It  shows  schedule  dates, 
staff  distribution,  required  facilities  and  machine  times.  Typically,  it  is  combined 
with  estimated  resource  cost  to  become  a  management  planning  tool. 
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2.2. 2. 3  Programming  Standards 


Well-defined  enforceable  standard  programming  practices  with  specific  emphasis  on 
those  which  require  the  production  of  structured  code.  Standards  to  be  considered 
include  comment  and  specification  statement  usage,  data  item  initialization, 
statement  ordering  and  common  storage,  data  item  and  array  conventions. 

2.2. 2. 4  Incremental  Development 

Definition  of  progressively  more  complete  increments  of  system  capability.  Each 
increment  or  release  provides  an  independently  testable  level  of  the  evolving 
system. 

2. 2. 2. 5  User  Involvement  Planning 

Create  and  maintain  a  user  interface  to  ensure  in-depth  involvement  of  users  and 
provide  timely  review  and  feedback  of  system  development  information.  As  a 
minimum,  there  should  be  explicit  definition  of: 

-  user  review  attendance 

-  user  feedback  at  critical  project  points 

-  documentation  of  important  user  opinions  and  recommendations 

2. 2. 2.6  Industry  and  Technical  Involvement  Plan 

Create  and  maintain  an  interface  with  a  wide  range  of  industry  and  technical 
experts  with  related  experience.  These  experts  review  and  critique  all  major 
technical  plans  and  give  advice  about  the  technical  feasibility  and  acceptability  of 
the  product. 
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2.2.2 .7  Documentation  Standards 


Detailed  outlines  for  each  of  the  deliverable  and  transition  documents.  The 
documentation  must  be  tailored  to  the  needs  of  the  audience/user.  Documentation 
should  be  thoughtfully  discussed  between  the  customer  and  developer  at  some  time 
prior  to  the  system  bid  and  a  documentation  plan  should  be  developed.  Simply  citing 
an  applicable  standard  or  Contract  Data  Requirements  List  (CDRL)  format  is  not 
enough  since  it  merely  highlights  the  government  administrator  and  not  a 
"manager." 

2.2.3  Project  Evaluation  and  Control 

2.2.3. 1  Software  Development  Tools 

Usage  of  general  purpose  or  specifically  designed  support  programs  at  various  points 
during  development  to  assist  requirements  analysis,  design,  code  generation,  debug¬ 
ging,  testing  and  documentation.  Generally,  the  tools  fall  into  two  areas: 

a.  Requirements  Definition  and  Analysis  Tools  -  Tools  to  aid  in 
developing  a  complete,  consistent,  unambiguous  specification  which 
can  serve  as  a  requirements  baseline.  A  given  specification  can  be 
analyzed  to  check  for  completeness  and  consistency  to  validate  its 
content  prior  to  design  and  coding. 

b.  Design  Representation  and  Testing  Tools  -  Tools  developed  to  aid  in 
the  specification,  analysis  and  validation  of  preliminary  and  detail 
design  activities. 

2. 2. 3. 2  Complete  Preliminary  Design 

Define,  document  and  baseline  a  complete  preliminary  design.  It  should  include 
identification  of  top-level  software  components,  data  base  definition,  scheduling  and 
interface  information,  and  critical  algorithm  analysis. 
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2.2. 3. 3  Software  Configuration  Management 


Establish  and  follow  formal  software  management  principles.  These  principles 
specify  a  formal  means  for  problem  and  change  reporting,  change  control  and 
version  control.  Accurate  and  complete  identification,  accounting,  control  and 
auditing  of  the  elements  needed  to  develop  a  software  system  are  all  part  of 
configuration  management. 

2.2. 3. 3.1  Change  Control  Board 

Responsible  for  the  disposition  and  authorization  of  proposed  changes.  The  Board 
verifies  change  request  compliance  with  contractual  requirements  and  assures  the 
identification,  evaluation  and  consideration  of  technical  reasons  for  the  change. 

2. 2. 3. 3. 2  Blockpoint  Change  Control 

A  change  control  device  for  indicating  when  the  approved  change  request  will  be 
completed  and  released.  The  project  manager  accumulates  the  change  activity  and 
determines  resource  availability,  work  priority  and  personnel  selection  for  each 
change. 

2. 2. 3. 4  Requirements  Specification  Breakout/Ba-'eline 

Develop,  review,  gain  customer  acceptance  and  document  the  software  require¬ 
ments  specification  to  be  used  as  a  formal  baseline.  The  approved  baseline  ensures 
mutual  understanding  and  formal  written  agreement  of  requirements. 

2. 2. 3.3  Technical,  Schedule  and  Cost  Reports 

Detailed  reports  submitted  regularly  (weekly,  monthly,  quarterly)  to  control  costs, 
scheduling,  and  make  project  progress  highly  visible.  These  reports  should  be 
automated  and  must  be  approved  and  signed  by  the  project  manager.  Reports 
include,  as  a  minimum,  estimated  versus  actual  costs  and  schedules. 
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2.2. 3.6  Independent  Test  Evaluation 


A  formal  organization  within  the  project  (but  separate  from  the  project's  design  and 
development  organization)  which  is  responsible  for  developing  test  strategy  and 
performing  an  independent  test  and  evaluation  of  software  parallel  to  system 
development.  Independent  testing  involves  analysis  of  software  requirements, 
planning,  preparation,  and  performance  of  tests,  review  of  test  results,  discrepancy 
reporting  and  retesting. 

2.2. 3.7  Project  Checkpoints/Milestones 

Established  at  each  software  development  phase  in  the  project  to  delimit  stages  of 
project  evolution  and  to  provide  a  means  for  control.  Checkpoints  or  milestones 
may  also  be  representative  of  production  of  major  documents  or  project  deliver¬ 
ables. 

2.2. 3.8  Checkpoint  Reviews 

A  formal  review  after  each  project  checkpoint  or  milestone  to  verify  that  work 
performed  is  satisfactory  and  follows  the  project's  quality  standards. 

2.2. 3.9  Status  Reviews 


An  internal  review,  conducted  on  a  periodic  basis  or  upon  demand,  which  discusses 
project  status  and  technical,  resource  or  logistical  decisions. 

2.3  IPAD  DEVELOPMENT  OBJECTIVES 

Examination  of  IPAD  documentation  relative  to  the  customer's  program  objectives, 
the  contractual  conditions  and  the  statement  of  work  has  identified  development/ 
management  objectives  for  the  project. 
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These  development/management  objectives  are  grouped  in  three  categories  which 
describe  the  nature  of  the  objectives.  Management  life  cycle  objectives  are  those 
which  concern  the  progress  and  status  of  the  project  work.  User  interface 
objectives  reflect  the  desire  to  establish  and  maintain  good  communication  between 
the  users/customers  and  the  project  team.  Objectives  of  technical  excellence 
reflect  concern  that  the  final  product  be  in  accordance  with  requirements, 
dependable,  reliable  and  of  overall  high  quality. 

2.3.1  Management  Life  Cycle  Objectives 

2. 3. 1.1  On  Schedule 

Achieve  intermediate  and  final  milestones  on  or  before  scheduled  completion  dates. 

2.3. 1.2  Under  Cost 

Accomplish  individual  and  cumulative  tasks  at  or  below  proposed  cost. 

2. 3. 1.3  Status  Visibility 

Provide  for  a  high  degree  of  visibility  into  IPAD  development  status. 

2. 3. 1.4  Problem  Recognition  and  Correction 

Provide  for  rapid  recognition  of  project  problems  and  equally  rapid  identification 
and  implementation  of  necessary  corrective  action 

2. 3. 1.5  Contractor  Commitment 

Demonstrate  high-level  contractor  commitment  to  the  IPAD  program. 


2.3.2  User  Interface  Objectives 


2.3.2. 1  User  Involvement 


Incorporate  a  high  degree  of  user  involvement  in  both  definition  and  periodic 
assessment  of  IPAD  system  features  and  functional  capabilities. 

2. 3.2. 2  Usefulness  Demonstration 

Provide  early  demonstration  of  the  ultimate  feasibility  and  general  utility  of  the 
IPAD  system. 

2. 3.2. 3  Satisfy  Diverse  Needs 

Provide  for  satisfaction  of  the  diverse  needs  of  the  broadest  possible  set  of  users 
(i.e.,  aerospace  vehicle  designers  of  the  1980's). 

2.3.3  Technical  Excellence  Objectives 


2.3.3. 1  Standard/Procedure  Compliance 

Demonstrate  strict  compliance  with  self-imposed  standard  practices  and  procedures. 

2. 3. 3. 2  Reliability  and  Dependability 

Ensure  maximum  reliability  an-  de'  lability  of  the  IPAD  system. 

2.3. 3.3  Configuration  Management 

Provide  for  organized,  effective  control  of  changes  to  the  IPAD  system 
configuration. 
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2.3. 3. 4  Machine  Independence 


Ensure  maximum  machine  independence  of  the  IPAD  software. 

2. 3. 3.5  Maintainability 

Ensure  maximum  maintainability  (easy  modifiability)  of  the  IPAD  system. 
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3.0  SURVEY  DESCRIPTION 

3.1  SURVEY  OBJECTIVES 

In  order  to  determine  the  actual  relationship  between  IPAD  objectives  and  the 
application  of  management  techniques,  a  survey  was  given  to  key  IPAD  manage¬ 
ment  and  technical  personnel.  There  were  two  purposes  of  the  survey.  We 
sought  first  to  determine  which  of  the  candidate  general  management  techniques 
(detailed  in  Section  2.2)  are  currently  being  applied  or  are  proposed  to  be  applied  to 
IPAD  and  secondly,  to  identify  specific  instances  in  which  the  management  tech¬ 
niques  applied  to  IPAD  were  effective  contributors  toward  achievement  of  the 
stated  objectives. 

3.2  SURVEY  TECHNIQUE 

The  survey  was  divided  into  three  parts.  The  first  part  contained  the  list  of 
development  objectives  detailed  in  Section  2.3.  The  participants  were  asked 
to  prioritize  the  objectives.  That  is,  within  the  context  of  IPAD,  they  were  asked 
to  identify  the  most  critical  objective,  next-most  critical  objective,  and  so  on. 

The  second  part  contained  a  list  of  the  candidate  management  techniques  which 
were  detailed  in  Section  2.2.  The  participants  were  asked  to  indicate  whether  they 
believed  the  techniques  were  currently  being  applied,  were  proposed  to  be  applied, 
or  were  not  planned  to  be  applied  to  IPAD. 
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The  third  part  of  the  survey/questionnaire  was  a  matrix  which  had  candidate 
management  techniques  on  the  horizontal  axis  and  development  objectives  on  the 
vertical  axis.  The  participants  were  asked  to  relate  the  objectives  to  the 
techniques  actually  used  to  accomplish  the  objectives.  If  the  technique  had  a  strong 
positive  effect  upon  achieving  the  objective,  the  participant  entered  an  S  in  the 
corresponding  matrix  element.  If  the  technique  had  only  a  moderately  positive 
effect  toward  objective  achievement,  the  participant  entered  an  M  in  the  matrix 
element.  If  the  participant  felt  that  the  technique  had  no  effect  (or  an  uncertain 
effect)  on  achievement  of  the  objective,  the  matrix  element  was  left  blank. 

3.3  SURVEY  RESULTS 

Survey  participants  prioritized  the  thirteen  objectives  on  a  scale  of  1  (high  priority 
objective)  to  13  (low  priority  objective).  The  results  were  summarized  into  three 
categories:  high,  medium  and  low  priority.  Responses  with  a  rank  from  1  to  k  were 
classified  as  high  priority,  5  to  8  as  medium  priority,  and  9  to  13  as  low  priority.  A 
summary  of  the  results  of  prioritization  appears  in  Appendix  A. 

Participants  next  gave  each  candidate  management  technique  a  score  which 
indicated,  to  the  best  of  the  participant's  knowledge,  the  technique's  application  to 
IPAD.  A  score  of  1  indicated  that  the  technique  was  currently  being  used.  A  score 
of  2  indicated  that  the  technique  was  not  currently  being  used,  but  was  proposed  for 
use  as  the  project  progressed.  Finally,  a  score  of  3  indicated  that  the  technique  was 
not  being  applied  and  was  not  proposed  to  be  applied  to  IPAD. 

The  results  of  this  part  of  the  survey  were  summarized  into  three  categories.  Tech¬ 
niques  with  a  score  of  1  or  2  were  grouped  together  since  they  represented 
techniques  which  were  applied  or  were  to  be  applied  to  IPAD.  The  number  of 
techniques  with  a  score  of  3  were  totaled.  In  addition,  some  participants  indicated 
that  they  had  no  personal  knowledge  of  the  technique  application.  These  indications 
were  summarized  under  a  third  category.  A  table  which  summarizes  the  results  of 
the  second  part  of  the  survey  appears  in  Appendix  B. 
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The  third  part  of  the  survey  indicated  whether  the  participant  felt  the  techniques 
had  a  strong  (S),  moderate  (M),  or  uncertain  (blank)  effect  toward  objective 
accomplishment.  A  summary  of  the  responses  is  presented  in  Appendix  C.  Each 
element  of  the  summary  matrix  has  been  divided  into  two  parts.  The  upper  part  of 
the  element  contains  the  total  number  of  strong  positive  responses,  while  the  lower 
part  reflects  the  total  number  of  moderate  positive  responses. 


<kO  SURVEY  RESULTS 

4.1  SURVEY -PARTI 

After  reviewing  the  survey  results  of  Part  1  (see  Appendix  A),  a  few  conclu¬ 
sions  can  be  made.  The  objectives  which  were  ranked  as  first,  second,  and  third  by 
the  participants  are  Satisfy  Diverse  Needs,  User  Involvement  and  Usefulness 
Demonstration.  It  is  important  to  note  that  these  three  objectives  were  cate¬ 
gorized  as  objectives  which  aid  the  user  interface.  As  stated  in  the  IPAD 
Requirements  document  "The  principal  objective  of  IPAD  is  to  provide  a  computer 
software  system  which  will  aid  the  U.S.  Aerospace  Industry  in  the  design  of  future 
vehicles."  From  this  statement,  and  other  evidence  of  emphasis  on  the  user 
interface,  it  is  apparent  that  the  participants  were  somewhat  biased  in  their 
prioritization  due  to  contractual  (NASA)  requirements.  That  is,  the  three  highest 
ranked  objectives  would  probably  receive  lower  ranks  if  surveyed  in  a  more  general 
atmosphere. 

The  objectives  of  Contractor  Commitment  and  Reliability  and  Dependability  were 
ranked  next  highest.  A  high  level  of  contractor  commitment  is  desirable  on  all 
software  projects.  As  one  participant  commented,  "it  is  implied  by  the  contractor's 
bid  and  must  therefore  be  ranked  very  high." 

Maintainability,  which  some  sources  estimate  consumes  70%  of  software  dollars,  is 
ranked  low  by  IPAD.  Again,  the  emphasis  on  technical  quality  supercedes  the 
concept  that  the  system  should  be  easily  modifiable.  The  contract  does 
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not  specifically  call  out  the  objective  of  maintainability,  it  does  call  for  a  product 
which  passes  certain  acceptance  tests. 

4.2  SURVEY  -  PART  2 

Part  2  of  the  survey/questionnaire  was  designed  to  determine  which  of  the  candi¬ 
date  management  techniques  (identified  in  Section  2.2)  are  being  applied  or  are 
proposed  to  be  applied  to  IPAD. 

Of  the  thirteen  survey  participants,  ten  responded  to  Part  2  of  the  survey. 
These  ten  participants  had  expertise  in  most  aspects  of  the  IPAD  program  and 
were  qualified  to  respond  to  Part  2.  Some  of  the  10  participants  felt  unqualified  to 
respond  to  elements  of  Part  2  of  the  survey.  These  responses  have  been  grouped 
separately.  Part  2  survey  results  have  been  summarized  in  Appendix  B. 

Sixteen  of  the  thirty-two  management  techniques  were  unanimously  identified  as 
having  current  or  proposed  IPAD  application.  In  addition,  ten  more  of  the 
techniques  were  unanimously  identified  as  being  applied  by  those  participants  with 
personal  knowledge  of  the  technique's  use.  The  technique  titled  "Work  Authori¬ 
zation  Form"  was  identified  by  all  but  one  participant  as  being  applied  on  IPAD. 
Survey  results  for  the  remaining  five  techniques  deserve  closer  analysis. 

Two  techniques,  "Egoless  Teams"  and  "Design  Representation  and  Testing  Tools," 
were  identified  as  being  applied  by  six  participants,  not  being  applied  by  two 
participants,  and  having  unknown  application  by  two  participants.  Due  to  lack  of 
positive  indication  of  the  use  of  these  techniques,  it  appears  that  they  are  not 
among  the  management  techniques  being  systematically  applied  to  IPAD.* 


*  During  research  for  Task  4,  a  technique  for  "Design  Representation  and  Testing 
Tools"  was  found  to  be  under  development  by  IPAD.  IPAD  evaluation  of  the 
usefulness  of  techniques  is  an  ongoing  activity  and  new  technology  is  constantly 
being  introduced  in  the  Software  Tools  area.  Therefore,  it  can  be  expected  to  find  a 
technique  under  development  which  was  not  a  part  of  the  original  project  plan. 
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The  technique  "Chief  Programmer  Teams"  had  an  almost  equal  number  of  response's 
in  each  of  the  three  categories.  Again,  due  to  lack  of  positive  indication  of  the  use 
of  this  technique,  we  must  assume  that  it  is  not  being  systematically  applied  on  the 
IPAD  program. 

The  two  techniques  which  remain  to  be  interpreted  are  "Personnel  Requirements 
Form"  and  "Requirements  Definition  and  Analysis  Tools."  These  techniques  have 
been  identified  by  the  majority  of  participants  as  techniques  which  are  not  being 
applied  to  IPAD. 

In  summary,  it  appears  that  twenty-eight*  of  the  original  thirty-two  management 
techniques  have  been  identified  as  having  current  or  proposed  IPAD  application, 
while  four*  techniques  are  not  being  systematically  applied  or  do  not  have  enough 
positive  evidence  to  indicate  application  on  IPAD. 

4.3  SURVEY  -  PART  3 

4.3.1  Modeling 

The  primary  objective  of  Part  3  of  the  survey/questionnaire  was  to  define  the 
relationship  between  the  stated  objectives  and  the  management  techniques  being 
applied  on  IPAD. 

Selection  of  a  modeling  method  to  evaluate  the  responses  to  the  survey/question¬ 
naire  involved  the  development  and  use  of  a  technique  to  measure  the 
objective/technique  relationship  identified  by  survey  participants.  A  mathematical 
algorithm  was  developed  to  summarize  and  analyze  the  response  data.  The 
algorithm  used  to  obtain  an  index  for  each  element  of  the  matrix  is: 


*  Including  Design  Representation  and  Testing  Tools 
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Index  =  Is  +  M 
T 


where: 

T  is  the  total  number  of  survey  participants 
S  is  the  number  of  strong  responses  for  the  element 
M  is  the  number  of  moderate  responses  for  the  element 

This  algorithm  evaluates  the  response  data  in  two  ways.  The  first  part  of  the 
algorithm  evaluates  the  strong  (S)  and  moderate  (M)  responses  for  each  matrix 
element  as  if  they  had  equal  weight.  The  second  part  of  the  algorithm  incorporates 
the  idea  that  strong  (S)  responses  have  much  more  (3  times  more)  weight  in  the 
analysis  than  moderate  (M)  responses. 

The  rationale  behind  the  second  part  of  the  algorithm  is  the  nature  of  the  S  and  M 
responses  themselves.  To  indicate  that  a  management  technique  has  a  strong  (S) 
effect  toward  accomplishment  of  an  objective,  the  participant  must  have  been 
firmly  convinced  of  the  existence  of  a  technique/objective  relationship.  However,  if 
the  participant  felt  that  the  technique  had  some  effect  on  the  objective,  but  was 
unsure  of  the  strength  of  the  effect,  he  would  be  likely  to  respond  that  the  effect 
was  moderate  (M).  In  light  of  the  nature  of  S  and  M  responses,  the  weight  of  S 
responses  should  be  greater  than  M  responses.  Through  iterative  use  of  different 
weight  schemes  on  assumed  survey  responses,  a  final  weighting  factor  of  3  was 
arrived  upon. 

4.3.2  Example 

The  iterative  method  for  determining  the  weight  of  strong  (S)  and  moderate  (M) 
responses  was  supported  by  assumed  survey  responses.  These  assumed  responses 
were  used  in  place  of  the  S  and  M  variables  in  the  algorithm  presented  in  Section 
4.3.1.  To  illustrate  the  algorithm,  these  assumed  responses  and  final  results  using 


the  algorithm  are  shown  in  this  section.  Figure  4.3.2. 1  shows  an  example  of  assumed 
responses  for  a  matrix  with  two  objectives  (Objectives  A  and  B)  and  three 
techniques  (Techniques  A,  B,  and  C).  In  this  example  the  sample  size  was  thirteen. 


Technique 

A 

Technique 

B 

Technique 

C 

S  -  6 

6 

10 

Objective  A 

M  -  4 

6 

3 

blank  -  3 

1 

0 

S  -  2 

0 

4 

Objective  B 

M  -  4 

13 

6 

blank  -  7 

0 

3 

Figure  4.3.2. 1 

EXAMPLE  SURVEY  RESPONSES 


Using  these  assumed  survey  responses,  the  algorithm  is  used  to  determine  an  index 
for  each  of  the  six  elements.  The  indices  produced  by  the  algorithm  are  shown  in 
Figure  4. 3.2. 2. 


Technique 

Technique 

Technique 

A 

B 

C 

Objective  A 

.43 

.57 

.85 

Objective  B 

.12 

.33 

.36 

Figure  4. 3. 2. 2 

INDICES  OF  OBJECTIVE/TECHNIQUE  RELATIONSHIP 


The  indices  indicate  that  Technique  C  has  the  most  significant  effect  on  Objective 
A.  This  outcome  is  not  surprising  since  10  participants  felt  that  the  technique  had  a 
strong  positive  effect  on  the  objective. 
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Observe  the  index  results  for  Objective  A/Technique  A  and  Objective  B/Technique 
C.  The  results  show  that  Objective  A/Technique  A  has  a  higher  index.  Although 
there  were  a  total  of  10  responses  in  the  S  and  M  categories  for  each  element,  the 
index  of  Objective  A/Technique  A  is  higher  due  to  the  weighting  of  S  responses  with 
a  factor  of  3. 

4.3.3  Summary  Survey  Results 

The  detailed  results  for  Part  3  of  the  survey  are  presented  in  Appendix  C.  There  are 
far  too  many  individual  elements  of  the  matrix  to  permit  detailed  discussion  of  the 
assessed  relationship  of  each  stated  objective  to  each  of  the  management 
techniques.  Elements  of  the  matrix  can  be  disregarded  due  to  lack  of  substantial 
positive  evidence  of  the  objective/technique  relationship.  As  an  element  filtering 
device,  all  matrix  elements  which  have  less  than  three  responses  in  both  the  S  and  M 
categories  can  be  disregarded.  In  addition,  five  columns  of  the  matrix  which 
represent  management  techniques  not  applied  on  IPAD  (see  Section  4.2)  can  be 
disregarded  from  Part  3  interpretation.  Elements  of  the  matrix  which  remain  after 
the  filtering  described  above  has  been  performed  are  then  evaluated  using  the 
algorithm  described  in  Section  4.3.1.  The  composite  indices  which  are  derived  from 
the  use  of  the  algorithm  are  presented  in  Appendix  D.  These  indices  range  in  value 
from  .03  to  .95. 

At  this  point  it  is  evident  that  response  data  which  does  not  indicate  a  strong 
objective/technique  relationship  remains  to  be  filtered  from  the  matrix.  To  do  this 
it  is  necessary  to  group  the  response  data  into  "domains."  The  following  domains 
were  selected  for  the  final  element  filtering: 

Response  Index  Domain  Data  Categorization 


0.0  -  0.14 
0.15  -  0.49 
0.50  -  1.0 


Inconclusive  relationship 
Moderate  Positive  relationship 
Strong  Positive  relationship 
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Looking  closely  at  these  domains,  it  may  appear  that  the  Moderate  and  Positive 
categories  have  a  wide  range  and  are  skewed  high.  For  example,  it  may  seem 
unreasonable  to  specify  that  indices  of  0.50  and  above  are  indicative  of  a  strong 
objective/technique  relationship.  Normally,  an  upper  domain  would  fall  in  the  70-80 
percentile  category.  However,  one  must  consider  the  effect  of  the  indexing 
algorithm  upon  the  response  data.  The  algorithm  specifies  that  two  fractions  must 
be  multiplied  together  to  obtain  the  composite  index.  When  two  fractions  are 
|  multiplied,  the  result  is  a  smaller  fraction.  For  instance,  if  the  first  fraction  in  the 

algorithm  (indicating  the  total  number  of  positive  responses  related  to  total 
participants)  was  relatively  high  -  .8  or  eighty  percent  -  and  the  second  fraction 
(which  weights  strong  responses)  was  also  .8,  the  composite  index  would  be  .64  or 
sixty-four  percent.  If  the  upper  domain  was  related  to  indices  of  .70  or  above,  this 
example  index  would  have  been  classified  as  moderate.  Intuitively,  the  relationship 
would  have  been  classified  as  strong.  For  this  reason,  the  domains  used  in  this 
analysis  were  adjusted  as  necessary  to  approximate  intuition  and  permit  the  lumping 
of  objective/technique  relationship  indices  into  the  broad  categories. 

The  final  filtered  results  of  Part  3  of  the  survey/questionnaire  are  presented  in 
Appendix  E.  Those  elements  which  fell  into  the  0.0  -  0.14  domain  are  indicated  by  a 
blank  and  therefore  make  the  strong  (S)  and  moderate  (M)  indices  readily  visible. 

As  Appendix  E  shows,  there  are  10  strong  and  65  moderate  objective/technique 
relationships.  The  strong  and  moderate  relationships  identified  occupy  75/416  or  18 
percent  of  the  original  matrix  of  objectives  and  techniques. 

The  number  of  strong  and  positive  elements  prohibits  detailed  discussion  of  each 
element  in  this  report.  However,  only  a  brief  glance  at  the  Part  3  summary 
identifies  some  particularly  interesting  objective/technique  relationships  which  are 
illustrated  below. 

A  few  management  techniques  were  identified  which  appeared  to  be  useful  toward 
achievement  of  many  objectives.  Those  with  the  highest  apparent  usefulness  were: 
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Project  Manager  Concept 
Software  Notebook 
Reviews  and  Audits 
User  Involvement  Planning 
Industry  and  Technical  Involvement 
Change  Control  Board 


In  addition,  some  of  the  development  objectives  were  identified  as  having  many 
management  tools  which  had  a  strong  or  moderate  effect  toward  achievement  of  the 
objective. 


Figure  4.3.3. 1  illustrates  the  objectives 
management  tools. 

Objective 

On  Schedule 
Under  Cost 

Usefulness  Demonstration 

Status  Visibility 

User  Involvement 
Configuration  Management 

Satisfy  Diverse  Needs 


which  are  strongly  effected  by  certain 

Strongly  Effected  by 

Project  Manager  Concept 
Project  Manager  Concept 
User  Involvement  Planning 
Industry  and  Technical  Involvement 
Project  Control  Room 
Reviews  and  Audits 
User  Involvement  Planning 
Change  Control  Board 
Blockpoint  Change  Control 
User  Involvement  Planning 


Figure  4.3.3. 1 

STRONG  OBJECTIVE/TECHNIQUE  RELATIONSHIPS 


Status  Visibility  was  identified  as  having  11  techniques  which  had  a  moderate  effect 
and  2  techniques  which  had  a  strong  effect. 


Configuration  Management  was  moderately  effected  by  4  techniques  and  strongly 
effected  by  2  techniques. 


Standard/Procedure  Compliance  was  moderately  effected  by  10  techniques. 

There  are  some  instances  in  which  the  effectivity  index  (i.e.,  the  observed 
objective/technique  relationship)  does  not  match  either  our  intuition  or  statements 
of  intent.  Such  instances  can  be  attributed  to  poor  survey  response  data,  ambiguous 
description  of  the  objective  or  technique,  or  simply  that  a  chosen  technique 
measures  up  as  less  powerful  than  advertised.  Investigation  of  these  instances  and 
resolution  of  the  cause  will  be  a  subject  of  subsequent  reports  on  the  findings  of  the 
Management  Tools  Case  Study. 
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5.0  INDIVIDUAL  RESPONSIBILITIES  AND  DIRECTION 

5.1  KEY  RESPONSIBILITIES 

To  identify  individuals  within  IPAD  with  key  functional  responsibilities,  a  list  of 
individuals  was  obtained  from  the  Management  Plan.  This  list  was  shown  to  the 
IPAD  Software  Development  Manager  to  ensure  that  all  individuals  had  been 
identified.  A  few  names  and  titles  were  added  and  a  draft  of  the  revised 
Management  Plan  was  provided  which  contained  textual  descriptions  of  key  respon¬ 
sibilities  in  planning,  controlling,  reviewing  and  assessing  project  activities.  It  is 
appropriate  to  define  what  the  terms  planning,  controlling,  reviewing,  and  as¬ 
sessing  mean  and  imply. 

Responsibilities  in  planning  are  concerned  with  devising  a  scheme  for  making,  doing 
or  arranging  project  activity.  Controlling  responsibilities  imply  regulating,  direct¬ 
ing  and  exercising  authority  over  the  course  of  project  activities.  Responsibilities 
in  reviewing  concern  the  formal  inspection  or  critical  evaluation  of  project  activi¬ 
ties.  Assessing  responsibilities  relate  to  judgment  of  the  worth,  importance  or 
value  of  project  activities. 

The  central  individuals  in  the  planning,  controlling,  reviewing  and  assessing  efforts 
are  the  Program  Manager  (PM)  and  the  Assistant  Program  Manager.  The  PM  is 
responsible  for  the  technical  approach  and  content  of  the  engineering  and  software 
products,  cost  and  schedule  control,  the  interface  between  the  development  team 
and  the  Industry  Technical  Advisory  Board,  and  for  total  IPAD  contract  perform¬ 
ance.  The  emphasis  of  the  PM  is  on  long-term,  general  direction  while  day-to-day 


functional  management  is  delegated  to  other  managers.  The  Assistant  Program 
Manager  supports  the  PM  through  complementary  (rather  than  duplicative)  responsi¬ 
bilities  which  are  of  the  near  term,  day-to-day  program  operations  nature.  The 
Assistant  Program  Manager  provides  overall  daily  technical  integration  and  coordi¬ 
nation,  assists  the  PM  in  review  preparation  (and  any  resulting  action  item  follow¬ 
up),  and  is  the  focal  point  for  all  contract  deliverables. 

The  PM  delegates  responsibility  and  authority  to  other  IPAD  managers.  The 
following  sections  contain  textual  descriptions  of  the  responsibilities  of  the  key 
IPAD  individuals.  These  descriptions  were  obtained  from  a  draft  of  the  revised 
Management  Plan  and  may,  therefore,  be  subject  to  minor  modifications. 

5.1.1  Business  Manager  (3M) 

The  Business  Manager  has  the  responsibility  of  preparing  all  monthly  and  quarterly 
schedule  and  cost  reports;  of  reviewing  cost  and  schedule  performance;  of  establish¬ 
ing  and  maintaining  the  IPAD  control  room;  and  of  interacting  with  Contract 
Administration  in  defining  and  negotiating  contract  questions.  He  has  the  author¬ 
ity  to  access  Boeing  Company  records  for  the  purpose  of  extracting  relevant 
cost  data  for  reporting  and  performance  review;  and  to  require  relevant  resource 
and  schedule  data  from  other  functional  group  heads.  He  will  assist  Program 
Support  in  the  purchase  or  rental  of  computing  facilities.  He  is  responsible  for 
defining  the  requirements  for  program  physical  facilities  and  coordinating  the 
installation  of  those  facilities. 

5.1.2  Industry  Technical  Advisory  Board  (1TAB)  Interface  Manager 

The  1TAB  Interface  Manager  has  the  responsibility  of  preparing  the  User  Involve¬ 
ment  Plan;  of  organizing  and  managing  the  functioning  of  ITAB;  and  to  correlate 
IPAD  developments  with  Boeing  engineering  and  manufacturing  organizations. 
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He  is  responsible  for  contacting  1TAB  representatives  for  the  purposes  of  organizing 
meetings,  communicating  technical  information,  and  conducting  critiques.  He  will 
also  contact  the  management  and  technical  personnel  of  IPAD  users,  as  directed  by 
NASA,  to  arrange  or  negotiate  communication,  training,  etc. 

5.1.3  Engineering  Development  Manager  (EDM) 

The  Engineering  Development  Manager  has  the  overall  responsibility  for  the 
engineering  tasks  on  the  IPAD  Program.  These  tasks  essentially  cover  user 
functional  requirements,  reviews  of  software  design,  software  acceptance  and 
training.  He  is  assisted  by  key  personnel  with  responsibilities  for  engineering 
process  definitions,  product  manufacturing  interface  definitions,  information  pro¬ 
cessing  requirements,  ana  user  functional  requirements.  New  responsibilities  will  be 
assigned  to  key  personnel  consistent  with  later  phases  of  the  program  involving  such 
tasks  as  acceptance  testing,  and  training.  He  will  work  with  the  ITAB  Interface 
Manager  in  corresponding  with  and  obtaining  reviews  by  ITAB  members  or  other 
members  of  the  using  community. 

He  has  the  authority  to  specify  and  prepare  with  NASA  concurrence,  or  as  directed 
by  NASA,  technical  computer  programs  and  data  bases  to  be  used  for  demonstration, 
installation  and  acceptance  testing;  to  accept  or  reject  software  specifications, 
designs  and  release  definitions  based  upon  their  compliance  with  the  engineering 
functional  requirements;  and  to  accept  or  reject  all  IPAD  software  installations 
based  upon  the  acceptance  tests.  The  integration  of  engineering  and  software 
technology  is  the  responsibility  of  the  Assistant  Program  Manager.  Conflicts 
between  engineering  and  software  development  will  be  resolved  by  the  Program 
Manager. 

5.1.4  Software  Development  Manager  (SDM) 

The  Software  Development  Manager  has  overall  responsibility  for  all  computing- 
related  activities  on  the  IPAD  project  including  organizing,  planning,  directing, 
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controlling,  and  reporting.  He  has  responsibility  for  the  cost,  schedule,  and  quality 
performance  of  all  computing  tasks  on  the  program. 

The  Software  Development  Manager  will  develop  the  organizational  structure  for 
the  systems  group  that  will  best  achieve  system  objectives  and  prepare  job 
descriptions  for  all  key  personnel  showing  clear  lines  of  responsibility,  decision¬ 
making  authority,  and  well  defined  accountability.  He  will  obtain  the  necessary 
staff  required,  coordinate  training  requirements  and  conduct  an  effective  training 
program,  monitor  personnel  performance  and  conduct  performance  reviews  for  the 
systems  staff,  and  coordinate  consultation  support,  as  required. 

He  will  ensure  that  planning  activities  are  conducted  so  that  tasks  and  activities 
have  clearly  defined  objectives,  task  dependencies,  priorities,  risks,  milestones,  and 
responsibilities  as  well  as  realistic  estimates  and  schedules  and  will  document  and 
present  such  plans  for  review  by  the  Program  Manager.  He  will  review  and  approve 
all  subordinate  plans  in  the  areas  of  user  involvement,  development  test,  acceptance 
test,  incremental  release,  training,  documentation,  and  configuration  and  control. 

The  Software  Development  Manager  will  initiate  policies  and  procedures  for  the 
performance  of  software  development  activities  including  planning,  estimating, 
design,  construction,  testing,  maintenance,  resource  usage,  training,  and  general 
administration.  He  will  ensure  that  the  systems  team  is  properly  motivated  to  deal 
with  their  responsibilities  and  with  personal  problems  that  might  interfere  with  the 
performance  of  their  duties.  This  responsibility  includes  handling  ail  communi¬ 
cations  external  to  the  systems  group  and  coordinating  the  systems  team's  required 
involvement  in  those  communications  to  minimize  the  impact  on  their  work  load. 

The  Software  Development  Manager  will  also  deal  with  all  administrative  re¬ 
sponsibilities  of  the  systems  organization  and  make  (or  obtain)  decisions  in  a  timely 
manner  to  keep  the  project  moving  without  undue  delays.  He  will  monitor  and 
review  the  IPAD  project's  progress  in  the  areas  of  cost,  schedule,  and  quality 
performance;  anticipate  and  identify  problems  and  incorporate  contingency  plans  to 
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deal  with  them;  and  evaluate  personnel  performance  and  take  corrective  action  as 
required. 


The  Software  Development  Manager  will  report  to  the  program  management  on 
manpower  and  skill  requirements,  schedule  and  resource  utilization,  technical  status 
of  milestones  and  deliverables,  and  problems  with  resulting  impacts  and  contingency 
actions  taken. 

5.1.5  Technical  Director/Chief  Designer  (CD) 

The  Technical  Director  is  responsible  for  the  design,  construction,  development 
testing,  maintenance,  and  documentation  of  the  IPAD  computing  system.  He  will 
ensure  that  these  items  are  correct,  compliant  with  the  contract  statement  of  work 
and  the  IPAD  requirements,  and  consistent  with  IPAD  software  development 
standards.  He  is  responsible  for  preparing  for  reviews  of  the  computing  technical 
work,  presenting  the  results  of  such  work  at  those  reviews,  and  for  responding  to 
action  items  and  recommendations  that  result  from  those  reviews. 

The  IPAD  Technical  Director/Chief  Designer  is  the  leader  of  the  IPAD  design  team, 
which  consists  of  the  IPAD  Systems  Integrator/ Assistant  Chief  Designer  and  the 
IPAD  System  Technical  Leaders  for  the  following  functions:  1)  Methodology; 
standards,  and  tools;  2)  Distributed  computing  system;  3)  Data  management  system; 
it)  User  interface;  5)  User  functions;  and  6)  Geometry/graphics. 

The  IPAD  Technical  Director/Chief  Designer  is  responsible  for  planning  technical 
tasks  under  his  control  or  delegating  such  tasks  to  the  IPAD  Systems  Integrator/ 
Assistant  Chief  Designer  or  to  the  appropriate  IPAD  System  Technical  Leader(s). 
He  directs  key  people  in  carrying  out  planned  tasks,  establishes  software  develop¬ 
ment  standards,  and  acquires  the  methods  and  tools  that  will  be  used  to  support  the 
IPAD  computing  tasks. 
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The  IPAD  Technical  Director/Chief  Designer  will  define  the  manpower  requirements 
of  the  computing  staff.  He  will  identify  training  and  technical  consultation 
requirements  and  commercial  sources  of  IPAD  system  software  components  and  will 
provide  assistance  in  completing  IPAD  software  development  tasks. 

The  IPAD  Technical  Director /Chief  Designer  (AD)  has  the  authority  to  approve  the 
computing  task  plans  and  end  items,  and  he  will  approve  computing  standards, 
methods,  and  tools  used  to  support  the  software  development  tasks.  He  is  directly 
accountable  to  the  IPAD  Software  Development  Manager  for  the  planning,  conduct, 
completion,  and  quality  of  all  IPAD  software  development  tasks.  He  reports  on  the 
status  of  IPAD  system  technical  tasks  to  the  IPAD  System  Development  Manager. 

5.1.6  IPAD  Systems  Integrator/ Assistant  Chief  Designer  (AD) 

The  IPAD  Systems  Integrator/Assistant  Chief  Designer  will  assist  the  Chief  Design¬ 
er  in  consolidating  all  of  the  IPAD  component  designs  into  a  total  system  design  at 
each  level  of  detail,  maintaining  a  log  of  design  decisions  and  unresolved  design 
problems,  and  identifying  task  interdependencies  between  the  development  teams. 

As  IPAD  Systems  Integrator,  he  will  primarily  be  responsible  for  technical  assist¬ 
ance  in  the  resolution  of  IPAD  system  problems,  drawing  upon  the  expertise  of 
computing  specialists,  engineers,  and  others  as  necessary. 

From  the  results  of  the  various  design  teams,  the  IPAD  Systems  Integrator  will 
ensure  production  of  a  complete,  consistent,  integrated  design.  He  will  continually 
review  1)  technical  designs,  especially  for  compatibility  with  identified  program/ 
SOW  requirements,  and  2)  design  and  documentation  for  conformity  to  applicable 
standards.  He  will  also  schedule,  coordinate,  and  conduct  technical  reviews  and 
work  sessions  relative  to  design  integration  and  their  problem  identification  and 
resolution. 
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The  IPAD  Systems  Integrator/Assistant  Chief  Designer  will  provide  technical 
support  in  the  form  of  review  responses;  coordinate  design  document  production; 
identify  and  coordinate  resolution  of  conflicts  between  design  and  documentation; 
develop  an  IPAD  user's  manual;  participate  in  software  design  and  development 
experiments;  assist  in  training  activities;  and  provide  support  to  the  IPAD  operations 
organization  as  appropriate. 

5.1.7  Technical  Leader— Methodology,  Standards  and  Tools  (TL) 

Under  the  direction  of  the  Technical  Director/Chief  Designer,  the  Technical 
Leader— Methods,  Standards,  and  Tools,  will  define  and  document  software  develop¬ 
ment  procedures,  standards,  and  general  software  engineering  methodology  for  the 
IPAD  project  and  will  devise  necessary  tools  and  aids  to  support  such  development. 
This  task  consists  of  the  subtasks  described  in  the  following  paragraphs: 

Establishing  standards  for  IPAD  software  development  documents— This  sub¬ 
task  involves  defining  the  content  and  format  of  the:  a)  IPAD  Requirements 
Document,  b)  Preliminary  Design  Document,  c)  IPAD  Detail  Design  Docu¬ 
ment,  and  d)  IPAD  Program  Document. 

Establishing  review  procedures  for  major  IPAD  software  development  tasks 
and  results— This  subtask  defines  the  expected  results,  material  to  be  review¬ 
ed,  procedure  for  conducting  a  review,  and  responsibilities  for  each  major 
review.  Procedures  apply  to  reviews  to  be  conducted  at  Boeing;  however,  they 
may  also  be  used  by  NASA  for  conducting  Intermediate  Design  Reviews  (IDR) 
and  Critical  Design  Reviews  (CDR).  These  IPAD  reviews  cover:  a)  Require¬ 
ments;  b)  Preliminary  design;  c)  Detail  design;  d)  Program  construction, 
testing,  and  evaluation;  e)  Program  and  documentation  acceptance  test;  and  f) 
Installation. 

Formulated  software  development  methods— This  subtask  will  formulate  and 
document  methods  to  be  used  in  defining,  developing,  describing,  testing, 
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evaluating,  installing,  and  maintaining  IPAD  software.  Such  methods  include: 
a)  Requirements  analysis;  b)  Preliminary  design;  c)  Detail  design;  d)  Program 
construction;  e)  Program  evaluation;  f)  Program  testing;  g)  Program 
installation;  and  h)  Program  maintenance. 

Identify,  acquire,  and  implement  software  development  tools  and  aids— This 
subtask  is  for  the  purpose  of  identifying,  acquiring,  and  implementing  tools  and 
aids  for  the  development  of  the  IPAD  program  when  limited  amounts  of  time 
and  resources  and  little  or  no  modifications  are  required.  Tools  and  aids 
include:  a)  Module  workbook,  b)  Coding  standards,  and  c)  Program 

modification  and  library  maintenance. 

Establish  software  development  reporting  standards  and  procedures— This 
subtask  will  define  the  content,  format,  and  application  of  the  following 
reports  to  be  published  by  the  IPAD  Software  Development  Group:  a) 
Technical  status,  b)  Technical  problems,  and  c)  Technical  bulletin. 

5.1.8  Program  Support  Manager 

The  Program  Support  Manager  has  the  responsibility  of  preparing  the  documentation 
and  configuration  control  plans,  managing  and  developing  the  dedicated  development 
computer,  packaging  software  for  delivery,  preparing  final  art  text  for  formal 
contract  documents,  and  for  establishing  and  maintaining  the  IPAD  Program  Support 
Library. 

5.1.9  IPAD  System  Technical  Leaders— Major  Components 

Under  the  direction  of  the  Technical  Director/Chief  Designer,  an  iPAD  System 
Technical  Leader  is  responsible  for  the  design,  construction,  development  testing, 
and  documentation  of  one  or  more  specific  major  IPAD  system  components,  as 
assigned.  Each  Technical  Leader  will  ensure  that:  a)  These  items  are  correct;  b) 
Designs  satisfy  their  requirements;  c)  Programs  satisfy  their  design  specifications; 


and  d)  Designs,  programs,  and  documents  produced  by  his  team  conform  to  IPAD 
Software  Development  Standards.  Each  Technical  Leader  is  responsible  for  planning 
the  tasks  of  his  team;  for  day-to-day  technical  problem-solving  assistance  to  his 
team;  for  seeking  assistance  from  the  Chief  Designer  in  such  problem-solving,  as 
necessary;  and  for  defining  manpower,  schedule,  and  training  requirements.  He  shall 
participate  in  the  preparation  and  presentation  of  technical  reviews. 

Each  IPAD  System  Technical  Leader  has  the  authority  to  carry  out  planned  tasks  in 
the  area  for  which  he  is  responsible  when  such  plans  are  approved  by  the  IPAD 
Technical  Director/Chief  Designer.  He  shall  approve  or  disapprove  the  designs, 
software  modules,  or  document  units  produced  under  his  direction. 

5.2  FORMAL  AND  INFORMAL  DIRECTION 

Project  communication  between  the  key  individuals  previously  identified  occurs 
through  formal  and  informal  channels.  The  formal  or  administrative  relationship 
that  the  individuals  communicate  through  is  shown  in  Figure  5.2-1. 

Notice  that  the  Program  Manager  and  the  Assistant  Program  Manager  are  depicted 
as  sharing  responsibilities  at  the  same  level.  This  separation  of  program  manage¬ 
ment  responsibilities  is  meant  to  focus  the  Program  Manager's  attention  on  general, 
long-term  program  direction,  financial/schedule  control  and  external  requirements. 
The  Assistant  Program  Manager  then  directs  the  daily  program  operations. 

The  Software  Development  Manager  and  the  Technical  Director  share  a  supportive 
role  also.  The  Software  Development  Manager  deals  with  all  the  administrative 
responsibilities  of  the  systems  group,  freeing  the  Technical  Director  to  deal  with 
design,  construction,  development  testing  and  maintenance  details. 

The  key  individuals  identified  in  Section  5.1  receive  formal  and  informal  direction 
when  fulfilling  their  designated  responsibilities. 


Figure  5.2-1  FORMAL  PROGRAM  ORGANIZATION  AND  REPORTING  RELATIONSHIPS 


Formal  direction  is  premeditated  and  usually  written  down.  It  consists  of  policies, 
procedures,  standards  and  communication  at  all  levels  (i.e.,  headquarters,  company, 
district  and  division  are  Boeing  levels)  which  must  be  followed  by  individuals  when 
carrying  out  functional  responsibilities. 

Informal  direction  may  be  written  down  but  is  usually  in  the  form  of  oral 
communication.  This  kind  of  direction  is  usually  a  result  of  a  problem  which  has 
occurred  and  was  not  anticipated,  therefore  no  formal  direction  was  specified. 
Generally,  problems  occur  and  a  solution  is  devised,  documented  and  implemented. 
Informal  direction  may  be  given  as  a  result  of  the  level  of  expertise  of  key 
individuals,  or  it  may  be  acquired  through  current  literature  in  the  field. 

Formal  and  informal  direction  was  investigated  via  the  Management  Plan  and 
through  conversations  with  some  of  the  individuals  with  key  responsibilities. 

Generally,  the  following  types  of  formal  and  informal  direction  were  identified: 

Formal  Direction 

•  Boeing  and  BCS  internal  procedures 

•  Boeing  and  BCS  operating  procedures 

•  Boeing  and  BCS  Contracts  Group 

•  Boeing  and  BCS  Finance  Group 

•  NASA  Reporting  Procedures,  Documentation  Standards 

•  IPAD  Statement  of  Work 

•  IPAD  Proposal 

•  IPAD  Feasibility  Study 

•  IPAD  Requirements 

•  Formal  reviews  of  both  documents  and  tasks  by  NASA,  IPO  and  ITAB 

•  Direction  through  lines  of  organizational  structure 

•  IPAD  Review  Procedures 

•  IPAD  Coding  Standards 


•  DOD  Industrial  Security  Regulations 

•  IPAD  Software  Development  Standards 

Informal  Direction 


•  Experience 

•  Training,  Lectures,  Demonstrations 

•  Informal  Reviews  of  documents  and  tasks  by  NASA,  IPAD  Project 
Office  (IPO),  and  ITAB 

•  Problem  Sessions,  self-study  workshops 

•  NASA  Technical  Bulletins 

•  NASA  documents  on  structured  programming  and  top-down  design 

•  SSDM  Phased-Development  Approach  methodology 

•  Current  literature  in  related  areas 

•  Standards  at  other  companies 

•  RADC  series  on  Structured  Programming 

•  American  National  Standards  Institute  (ANSI)  Standards 

•  IPAD  Team  Members 

•  Comparison  with  similar  historical  project  activities 

•  Computing  Specialists 

•  Boeing  Data  Center  Specialists 

Each  key  individual  uses  formal  and  informal  direction  in  a  different  manner.  The 
following  paragraphs  discuss  the  way  each  of  the  individuals  identified  earlier  use 
formal  and  informal  direction. 

The  Business  Manager  relies  on  formal  direction  from  the  Boeing  Finance  and 
Contracts  groups  to  determine  the  content  and  frequency  of  various  cost  and 
schedule  reports.  NASA  Reporting  Procedures  and  documentation  standards  also 
provide  formal  direction  of  report  content  and  format.  When  defining  physical 
facility  requirements  he  is  assisted  by  formal  Boeing  and  BCS  operating  procedures 
which  apply  to  lease/buy  decisions. 
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Informally,  the  Business  Manager  is  directed  by  his  own  experience  and  training  as 
well  as  that  of  other  IPAD  team  members. 


The  ITAB  Interface  Manager  receives  formal  direction  from  the  IPAD  Proposal  and 
the  IPAD  Statement  of  Work.  These  documents  define  the  format  and  content  of 
the  User  Involvement  Plan,  the  number  and  types  of  ITAB  members  as  well  as 
functional  responsibilities  of  ITAB  members. 

Informally,  he  relies  upon  his  experience  and  that  of  IPAD  team  members  and 
Boeing  to  determine  the  ITAB  members  and  to  conduct  ITAB  reviews. 

The  Engineering  Development  Manager  receives  formal  direction  through  the  IPAD 
Feasibility  Study,  the  IPAD  Statement  of  Work  (SOW),  IPAD  Requirements,  formal 
reviews  by  ITAB,  IPO  and  NASA,  BCS  and  Boeing  Commercial  Airplane  Company 
(BCAC)  interna]  procedures.  Each  of  these  kinds  of  formal  direction  assist  him  in 
determining  and  complying  with  the  functional  requirements  of  IPAD,  conducting 
software  design  reviews,  and  software  acceptance  and  training. 

BCAC  specialists  and  documentation  provide  informal  direction  when  establishing 
the  product  manufacturing  interface  definitions.  The  content  and  format  of  training 
materials  is  informally  directed  by  the  experience  of  the  Engineering  Development 
Manager  and  other  members  of  the  IPAD  team  as  well  as  that  of  other  experts  in 
the  field. 

The  Software  Development  Manager  relies  on  BCS  and  BCAC  procedures  and 
policies  as  well  as  the  SOW  and  Proposal  when  defining  the  organizational  structure 
of  IPAD.  When  reviewing  the  cost  and  schedule  of  project  activities  he  relies  on  the 
formal  assistance  of  the  Business  Manager  and  the  Boeing  Finance  group. 


Some  facets  of  organizational  structure  development  are  informally  directed  by  the 
experience  and  judgement  of  the  Software  Development  Manager.  He  also  relies  on 


his  experience  and  that  of  other  members  of  the  IPAD  when  initiating  policies  and 
procedures  in  planning,  estimating  and  design. 

The  Technical  Director/Chief  Designer  uses  the  IPAD  Requirements,  SOW  and 
established  standards  as  formal  direction  to  ensure  that  items  of  the  computing 
system  are  correct  and  satisfy  requirements.  Policies  established  by  IPAD  set 
formal  guidelines  for  review  content  and  format. 

Informally,  past  experience  of  the  IPAD  managers  and  the  IPAD  team,  as  well  as 
current  related  literature  and  NASA  and  BCS  guidelines  aid  in  the  development  of 
coding  standards  and  development  testing. 

Formal  direction  to  the  Systems  Integrator  is  provided  to  consolidate  the  design 
components  and  resolve  problems  by  the  IPAD  Management  Plan,  SOW  and  IPAD 
requirements.  A  design  representation  and  analysis  tool  is  available  to  assist  testing 
of  the  design  for  completeness  and  consistency,  however,  use  of  this  tool  is  not 
required.* 

The  Technical  Leader  receives  formal  direction  from  the  Chief  Designer  and  the 
NASA  IPAD  project  office  in  the  development  of  standards,  software  development 
procedures  and  the  software  engineering  methodology. 

A  good  deal  of  his  work  is  informally  directed  through  reference  to  NASA  design 
standards  documents,  BCS  Systematic  Software  Development  and  Maintenance 
(SSDM),  current  literature  in  the  field,  military  standards,  standards  from  other 


•During  initial  research  the  tool  was  under  development,  however,  development  of 
the  tool  was  discontinued.  A  preliminary  version  of  the  tool  which  serves  to 
document  the  design  and  provide  some  rudimentary  analysis  capabilities  is  available 


for  IPAD  use. 


companies,  Boeing  Standards,  NASA  Technical  Bulletins  and  the  RADC  series  on 
Structured  Programming.  In  addition,  standards  established  by  the  American 
National  Standards  Institute  (ANSI)  are  informally  used. 


The  Program  Support  Manager  is  formally  directed  by  the  IPAD  SOW,  NASA  docu¬ 
mentation  Standards,  the  IPAD  Feasibility  Study,  Boeing  Standards,  DOD  Industrial 
Security  Regulations  and  the  IPAD  Contract  Schedule  in  preparation  of  the 
documentation  and  configuration  control  plans.  BCS,  BCAC,  Boeing  Data  Center, 
and  NASA  procedures  for  configuration  control  and  program  support  libraries  are 
used  when  applicable. 

The  Program  Support  Manager  relies  heavily  on  his  many  years  of  experience  in 
configuration  control  and  program  support.  He  has  an  established  rapport  with  the 
Boeing  Data  Center  as  well  as  with  other  Boeing  individuals  whom  he  relies  upon  for 
knowledge  and  aid  in  problem  correction. 

The  System  Technical  Leaders  are  formally  directed  by  the  Technical 
Director/Chief  Designer.  They  rely  upon  the  SOW  and  IPAD  Requirements  to  ensure 
that  the  design  satisfies  the  contractual  requirements.  The  standards  developed  by 
the  Technical  Leader  of  Methodology,  Standards  and  Tools,  are  used  by  the  other 
Technical  Leaders  to  determine  conformance  to  project  standards. 

When  defining  manpower,  schedule  and  training  requirements  for  a  major  IPAD 
system  component,  the  Technical  Leaders  rely  on  their  experience  as  well  as 
comparison  with  similar  historical  project  activities.  An  automated  tool  (called 
Project  Control/70)  which  contains  records  of  previous  project  schedule  and 
manpower  requirements  is  informally  relied  upon  to  compile  component  resource 
requirements.  In  addition,  all  members  of  the  component  development  team 
participate  in  the  planning  process.  This  participation  ensures  accurate  estimation 
and  allows  other  team  members  to  gain  valuable  experience. 
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5.3  RELATION  BETWEEN  DIRECTION  AND  PROGRAM  OBJECTIVES 

To  verify  the  research  into  formal  direction,  dependence  on  informal  direction,  and 
to  initiate  determination  of  the  relationship  between  direction  and  the  program 
objectives,  we  formulated  three  hypothetical  scenarios  illustrative  of  typical 
problems  and  situations  often  encountered  in  major  software  development  efforts. 
Briefly,  these  hypothetical  scenarios  depicted: 

a)  A  requirements  change  during  a  Critical  Design  Review  (CDR). 

b)  A  design  deficiency  discovered  by  the  IPAD  Technical  Advisory  Board 
(1TAB). 

c)  Concern  about  an  acceptable  measure  of  testing  sufficiency  during  a 
Preliminary  Design  Review  (PDR). 

The  scenarios  have  been  more  fully  described  in  Appendix  F,  Hypothetical  IPAD 
Situation  Scenarios. 

These  hypothetical  scenarios  were  distributed  to  the  IPAD  Software  Development 
Manager  for  study  and  resolution.  The  Software  Development  Manager  consulted 
IPAD  team  members  who  would  be  responsible  for  scenario  resolution  and  produced 
lists  of  steps  which  would  be  taken  to  resolve  the  hypothetical  scenarios.  These  lists 
have  been  reproduced  in  Appendix  G,  IPAD  Scenario  Resolution. 

The  scenarios  were  illustrative  of  direction  received,  responsibilities  fulfilled  and 
actions  taken  to  resolve  hypothetical  problems.  They  provided  evidence  of  a  strong 
relationship  between  formal  and  informal  direction  and  IPAD  team  member  respon¬ 
sibilities. 

The  resolution  of  the  scenarios  did  not  give  evidence  of  a  strong,  explicit 
relationship  between  the  program  objectives  and  formal  and  informal  direction.  An 
all-encompassing  methodology  which  attempts  to  show  that  a  specific  form  of 
formal  or  informal  direction  will  accomplish  a  particular  program  objective  cannot 
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be  derived.  There  exist  components  of  formal  and  informal  direction  which  cannot 
be  predefined  and  are  therefore  not  measurable.  Examples  of  such  components  are 
personalities  and  individual  experience  and  capability. 

However,  the  scenarios  were  illustrative  of  the  manner  in  which  dispatching  of 
responsibilities  relates  to  the  achievement  of  program  objectives.  In  essence, 
corrective  action  (to  resolve  the  scenarios)  occurred  through  the  dispatching  of 
responsibilities.  In  this  way,  satisfaction  of  program  objectives  can  be  achieved  by 
expediting  responsibilities. 

To  provide  an  illustration  of  the  manner  in  which  responsibilities  satisfy  program 
objectives,  the  scenarios  were  used  in  conjunction  with  definitions  of  responsibilities 
set  forth  by  Robert  D.  Melcher  in  a  paper  entitled  "Roles  and  Relationships: 
Clarifying  the  Manager’s  Job."  Melcher  defines  types  of  managerial  responsibility: 

A.  General  Responsibility  -  The  individual  guides  and  directs  the  execution  of 
the  function  through  the  person  delegated  operating  responsibility. 

B.  Operating  Responsibility  -  The  individual  is  directly  responsible  for  the 
execution  of  the  function. 

C.  Specific  Responsibility  -  The  individual  is  responsible  for  executing  a 
specific  or  limited  portion  of  the  function. 

D.  Consulted  -  The  individual,  if  the  decision  affects  his  area,  must  be  called 
upon  before  any  decision  is  made  or  approval  is  granted,  to  render  advice 
or  relate  information,  but  not  to  make  the  decision  or  grant  approval. 

E.  Notified  -  The  individual  must  be  notified  of  action  that  has  been  taken. 

F.  Approve  -  The  individual  (other  than  persons  holding  general  and  operating 
responsibility)  must  approve  or  disapprove. 

A  matrix  was  constructed  which  depicted  the  scenario  corrective  steps  (from 
Appendix  G)  vertically  and  key  IPAD  individuals  horizontally.  For  each  step,  the 
individuals  having  the  types  of  managerial  responsibilities  described  above  were 
identified.  Next,  each  corrective  step  was  assumed  to  have  been  initiated  to 
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achieve  one  or  more  of  the  thirteen  program  objectives  identified  earlier.  Objec¬ 
tives  which  were  associated  with  the  corrective  steps  were  identified  in  the  right¬ 
most  column  of  the  matrix.  Figure  5.3-1  illustrates  the  matrix  representing 
Scenario  1. 

In  this  case,  the  first  step  in  scenario  resolution  was  IPAD  Program  Manager's  (PM) 
notification  of  the  Critical  Design  Review  action  item.  The  PM  has  specific 
responsibility  to  direct  the  overall  execution  of  tasks  in  order  to  accomplish  the 
requirements  change  hypothesized  by  Scenario  1.  The  NASA  IPAD  Project  Office 
(IPO)  must  approve  the  action  item  which  resulted  from  the  supposed  CDR. 
Program  objectives  4  and  12,  Status  Visibility  and  Contractor  Commitment, 
respectively,  are  supported  by  Step  1.  By  notifying  the  PM  and  the  NASA  IPO,  a 
high  degree  of  visibility  is  afforded  to  the  action  item.  PM  notification  provides 
contractor  (BCS)  commitment  at  a  very  high  level. 

Each  of  the  steps  for  processing  the  action  item  associated  with  Scenario  1  has  been 
evaluated  in  a  like  manner  and  is  documented  in  Figure  5.3-1.  Scenario  2  steps  were 
likewise  evaluated  and  the  results  appear  in  Figure  5.3-2. 

The  use  of  scenario  resolution  steps  in  conjunction  with  definitions  of  managerial 
responsibilities  defined  by  Melcher  has  helped  to  show  the  strong  relationship  of 
responsibilities  to  program  objectives. 

Study  of  the  development/management  objectives  supported  by  the  functional 
events  of  all  scenarios  as  depicted  in  Figures  5.3-1  and  5.3-2  will  show  that  not  all 
the  objectives  were  supported  by  the  set  of  steps.  This  occurs  due  to  the  situations 
depicted  by  the  scenarios.  That  is,  the  objectives  not  listed  (Nos.  1,  2,  3,  6,  7,  11 
and  13)  would  be  supported  by  scenarios  depicting  different  situations.  For  example, 
a  scenario  concerning  the  assessment,  evaluation  and  replanning  of  a  scheduled  WBS 
task  would  support  objectives  1  and  2  (among  others). 
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RESPONSIBILITIES  FOR  SCENARIO 


5.3-2  RESPONSIBILITIES  FOR  SCENARIO  2 


It  should  also  be  pointed  out  that  the  use  of  Robert  Melcher's  definitions  of 
responsibilities  is  a  very  limited  use  of  part  of  a  tool  he  has  developed  called  the 
Management  Responsibility  Guide  (MRG).  The  MRG  is  a  planned  and  systematic 
approach  to  group  and  related  managerial  functions  in  a  manner  to  aid  an 
organization  and  objectively  solve  its  organizational  problems. 


6.0  STATUS,  PROGRESS  AND  PERFORMANCE  INDICATORS 

6.1  GENERAL  INFORMATION 

Task  4  calls  for  determining  (through  key  people  and  by  studying  the  Management 
Plan)  what  indicators  IPAD  is  using  to  assess  status,  progress  and  performance.  It 
is  appropriate  to  define  what  the  terms  status,  progress  and  performance  mean  and 
imply. 

Status  is  a  set  of  circumstances  or  attributes  which  characterize  a  person  or  thing 
at  a  particular  time.  This  imples  that  a  status  indicator  is  that  which  provides 
explicit  evidence  of  the  state  or  condition  of  the  IPAD  program. 

To  progress  means  to  move  forward  or  onward;  to  advance  toward  completion  or  a 
higher  state.  This  has  several  implications:  first,  that  there  exists  a  thing  which  is 
uncompleted;  second,  that  progress  can  be  measured  as  an  increment  or  change  in 
the  status  of  something.  Progress  indicators  of  the  IPAD  program  give  evidence  of 
the  change  in  status. 

Performance  is  the  execution,  accomplishment  or  fulfillment  of  something  or  some 
task.  To  perform  a  task  implies  some  sort  of  operation  or  functioning,  usually  with 
regard  to  effectiveness.  That  is,  the  ingredients  of  performance  are  productivity 
as  well  as  worth  or  value.  Performance  indicators  on  IPAD  give  evidence  of  the 
efficient  consumption  of  effort  to  accomplish  a  task  effectively  and  with  a  degree 
of  technical  excellence. 
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6.2  MONITORING  STATUS,  PROGRESS  AND  PERFORMANCE 


Monitoring  of  status,  progress  and  performance  is  aided  through  the  use  of  the  Work 
Breakdown  Structure  and  the  Work  Authorization  Form.  The  principle  program 
management  control  mechanism  is  the  Work  Breakdown  Structure  (WBS)  and  the 
monitoring  and  review  of  design  and  development  technical  performance  relative  to 
planned  schedule  and  cost  of  work  breakdown  items.  The  following  figure,  copied 
from  the  IPAD  Management  Plan,  indicates  the  pervasive  use  of  the  WBS  in  IPAD. 
As  Figure  6.2-1  illustrates,  the  WBS  is  the  basis  for  scheduling,  management  by  the 
project  organization,  recording  of  costs  incurred  and  project  reporting.  The  formal 
means  for  delegating  work  in  the  IPAD  Program  environment  is  the  Work  Authori¬ 
zation.  Work  authorization  consists  of  three  things:  the  assignment  of  responsi¬ 
bility,  granting  of  authority,  and  accounting  for  results.  Formal  accountability  is 
achieved  through  use  of  the  Work  Authorization  form.  A  copy  of  the  current  IPAD 
Work  Authorization  (WA)  form  is  shown  in  Figure  6.2-2.  The  WA  describes  work  to 
be  performed  for  each  WBS  unit.  When  an  authorization  for  a  particular  level  of 
the  WBS  is  issued,  all  supporting  tasks  are  implicitly  authorized. 

6.3  IPAD  INDICATORS 

During  research  into  IPAD  indicators  of  status,  progress  and  performance  it  became 
increasingly  apparent  that  most  of  the  indicators  were  provided  through  the  use  of 
the  management  tools  and  techniques  discussed  previously.  This,  however,  was  not 
surprising  since  the  tools  were  chosen  (by  IPAD)  for  the  aid  they  would  provide  in 
the  areas  of  project  administration,  planning,  evaluation  and  control. 

The  list  which  follows  describes  indicators  which  IPAD  uses  to  assess  status, 
progress  and  performance.  When  the  indicator  is  provided  through  the  use  of  a  tool, 
the  tool  name  is  supplied  and  the  list  of  tools  (Section  2.2)  should  be  cross- 
referenced  since  the  tool  description  contains  a  general  description  of  the  indicator. 
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SU'/MARV  SCHEDULE  OF 


gure  6.2-1  WBS  AS  BASIS  OF  IPAD  PLANNING  AND  CONTROL 


ROJECT: 


1.  The  Project  File,  as  a  centralized  base  of  project  information,  contains  status, 
progress  and  performance  indicators.  Status  indicators  are  in  the  form  of 
reports,  correspondence,  documentation  of  findings,  conclusions,  recommen¬ 
dations  and  change  requests.  Progress  indicators  are  minutes  of  reviews  and 
meetings,  and  reports.  Performance  indicators  can  be  found  in  documentation 
of  findings,  conclusions,  and  recommendations,  minutes  of  reviews  and  meet¬ 
ings,  worksheets  and  other  analyses,  project  performance  data  and  documen¬ 
tation  on  problems  and  lessons  learned. 

2.  The  Software  Module  Workbook  (also  called  the  Software  Notebook)  of  IPAD 
contains  two  kinds  of  data;  that  which  will  be  included  in  design  documents, 
and  that  which  is  included  to  provide  management  with  an  understanding  of 
implementation  status.  Design  document  information  includes  module  pro¬ 
gram  listings,  figures  and  diagrams.  Status,  progress  and  performance 
visibility  is  provided  through  the  following  types  of  information: 

•  Walk-through  Approval  Sheets 

•  Implementation  Plans 

•  Test  Results 

•  Discrepancy  Reports 

Walk-through  Approval  Sheets  are  the  result  of  the  structured  walk-through 
and  certify  that  the  module  being  reviewed  is  either  accepted  or  to  be  revised. 
The  Approval  Sheets  provide  quick  visibility  into  module  review/acceptance 
status.  A  copy  of  the  Walk-through  Approval  Sheet  is  shown  in  Figure  6.3-2, 
following  Item  No.  5,  Walk-through. 

Implementation  Plans  contain  Work  Authorization  forms  filled  out  as  part  of 
component  planning  as  well  as  documentation  used  to  generate  estimates  and 
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schedules. 


Test  Results  consist  of  a  test  summary,  test  results,  and  test  case  input 
sections.  Each  test  is  described  (complete  with  environment  specification), 
the  results  of  each  case  are  recorded,  and  correspondence  of  results  with 
acceptance  criteria  are  noted.  The  design  team  manager  uses  the  Software 
Module  Workbook  to  monitor  module  status,  progress  and  performance. 

Discrepancy  Reports  are  used  when  a  problem  has  been  found  in  modules  which 
have  been  placed  under  configuration  control.  A  sample  Software  Problem 
Report  is  shown  in  Figure  6.3-6,  following  Item  No.  12,  Software  Configuration 
Management.  Although  not  in  use  by  IPAD  at  the  present  time,  a  Software 
Notebook  generally  contains  a  Summary  Cover  Sheet  like  that  shown  in  Figure 
6.3-1.  As  the  figure  illustrates,  the  status  of  a  module  can  easily  be 
determined  by  glancing  at  the  Cover  Sheet. 

3.  The  Support  Library  contains  an  official  copy  of  the  latest  version  of  the 
program.  The  status  of  software  construction  can  be  determined  by  obtaining 
the  number  of  modules  stored  within  the  library.  Since  the  library  contains 
archive  program  data,  comparison  of  current  and  archive  data  will  provide 
evidence  of  progress. 

4.  The  Project  Control  room,  as  a  tool  for  promoting  good  communications, 
provides  evidence  of  status,  progress  and  performance  in  relation  to  plan. 
Usually  the  IPAD  project  room  contains  displays  on  detailed  schedules,  budget 
cost  charts,  delinquent  items,  action  items,  project  technical  performance, 
status  assessment,  milestone  schedules  and  critical  events.  The  Project  Room 
is  the  central  location  of  evidence  of  status,  progress  and  performance 
indicators. 

5.  A  walk-through  can  provide  an  indication  of  programming  quality  (perform¬ 
ance)  as  errors,  discrepancies,  exposures  and  inconsistencies  may  be  identified 
while  the  programmer  "walks"  the  other  group  members  through  the  module 
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Figure  6.3-1  SAMPLE  SOFTWARE  NOTEBOOK  COVER  SHEET 
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logic  and  data  flow.  It  also  provides  a  measure  of  progress  since  it  represents 
completion  and  review  of  a  component. 

IPAD  Structured  Walk-throughs  follow  specified  rules  or  procedures  and  each 
participant  has  a  designated  role/function  which  he  must  fulfill.  The  result  of 
an  IPAD  Structured  Walk-through  is  a  Walk-through  Report  (also  called  a 
Walk-through  Approval  Sheet).  A  sample  Walk-through  Report  is  shown  in 
Figure  6.3-2.  The  Report  documents  the  participants'  responsibilities,  the 
review  agenda  and  review  decision.  The  review  may  result  in  an  action  list  of 
items  to  be  resolved.  By  viewing  the  Walk-through  Reports,  the  various  IPAD 
team  leaders  have  evidence  of  status,  progress  and  performance. 

Inspections  provide  performance  indication  as  the  program  product  is  verified 
for  compliance  with  requirements  and  acceptance  criteria.  Since  inspections 
are  a  means  to  capture  statistics  about  detected  errors,  a  collection  of 
inspection  documentation  should  provide  a  good  indication  of  project  perform¬ 
ance. 

Quality  assurance  reviews  and  audits  provide  evidence  of  performance  since 
they  test  for  satisfaction  of  requirements,  conformance  to  specifications  and 
adequacy  of  development  and  independent  testing.  For  example,  each 
document  is  subject  to  rigorous  quality  assurance  control.  A  QA  Check  List 
accompanies  the  document  for  review  by  the  manager  responsible  for  docu¬ 
ment  development,  the  Quality  Assurance  and  Configuration  Control  Manager, 
the  Assistant  Program  Manager,  and  the  Program  Manager.  A  copy  of  this 
form  is  presented  in  Figure  6.3-3.  This  form  also  depicts  the  document  review 
status  at  any  time  during  the  review  cycle.  A  manager  is  able  to  determine 
whether  approval  has  been  granted  by  each  of  the  other  responsible  managers 
by  the  presence  or  absence  of  signatures  in  the  lower  right-hand  corner. 
Another  checklist  is  used  for  quality  assurance  and  configuration  control.  The 
form  is  a  50-item  checklist  of  tasks  which  must  be  completed  for  each 
document  produced  by  IPAD.  This  form  is  presented  in  Figure  6.3-5 
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Coordinator 


Project 

Coordinator's  Checklist: 

1.  Confirm  with  producer(s)  that  material  is  ready  and  stable  _ 

2.  Issue  invitations,  assign  responsibilities,  distribute  materials 


Date 
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Agenda: 

1.  All  participants  agree  to  follow  the  (same)  set  of  rules. 

2.  New  project:  walkthrough  of  material 

Old  project:  item-by-item  checkoff  of  previous  action  list 

3.  Creation  of  new  action  list  (contributions  by  each  participant) 

4.  Group  decision 

5.  Deliver  copy  of  this  form  to  project  management 

Decision  Accept  product  as-is 

_ Revise  (no  further  walkthrough) 

_ Revise  and  schedule  another  walkthrough 


Signatures: 


Figure  6.3-2 


SAMPLE  WALKTHROUGH  APPROVAL  REPORT 


Figure  6.3-3  SAMPLE  QUALITY  ASSURANCE  CHECK  SHEET 


following  item  12  on  configuration  control  indicators.  Documentation  quality 
assurance  status  can  be  obtained  quickly  by  scanning  the  column  entitled 
"COMPLETED  BY"  on  this  form. 


IPAD  programmers  and  designers  must  follow  a  detailed  set  of  programming/ 
design  standards.  Among  the  design  standards  is  the  requirement  that  the 
system  be  developed  using  top-down  design.  Top-down  techniques  are  followed 
and  the  problem  is  reduced  into  a  set  of  components.  IPAD  components  have 
specified  characteristics.  For  example: 

a)  It  implements  a  defined  set  of  functions  required  ot  implied  by  IPAD 
software  specifications. 

b)  The  executable  and  comment  statements  (of  the  corresponding  procedure) 
can  be  listed  contiguously. 

c)  The  statements  are  enclosed  by  identifiable  boundaries. 

d)  The  component  can  be  referenced  from  other  parts  of  the  program  only  by 
its  name  or  its  single  entry. 

e)  The  component  will  have  a  single,  common  entry  and  a  single,  common 
exit. 

f)  The  component  can  reference  other  components,  suspend  its  execution 
upon  encountering  procedure  statements,  and  resume  execution  with  the 
next  immediate  statement. 

g)  All  called  components  must  return  to  their  caller  at  the  statement 
immediately  following  the  reference. 


While  the  seven  component  design  standards  are  not  the  complete  set  of  design 
standards,  they  are  illustrative  of  the  conventions  which  must  be  followed. 
Standards  such  as  these  provide  a  means  for  evaluating  design/programming 
performance. 

9.  User,  industry  and  technical  individuals  review  the  progress  and  performance 
of  IPAD.  They  participate  in  the  review  and  assessment  of  such  data  as: 

a)  User  Requirements 

b)  System  interface  plans/specifications 

c)  Software 

d)  Hardware  Plans 

e)  Technical  Plan- 

f)  Training  Plans 

g)  Test  Plans 

h)  Test  Results 

i)  Development  Schedules 

j)  User  Documentation 

k)  User  Evaluation  Results 

The  minutes  (or  reports)  which  result  from  user,  industry  or  technical  review 
are  evidence  of  IPAD  progress  and  performance. 

10.  Informal  auditing  of  conformance  to  documentation  standards  provides  an 
indication  of  the  performance  of  the  documentation  development  groups.  The 
IPAD  Documentation  Plan  specifies  the  standards  to  be  followed  by  the 
various  development  groups.  It  also  contains  outlines  of  the  planned  contents 
of  each  document.  Formal  Quality  Assurance  and  Configuration  Control  (see 
items  7  and  12)  which  is  performed  on  each  document  provides  evidence  of 
adherence  to  these  standards  thereby  yielding  indicators  of  performance. 
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The  Documentation  Plan  references  applicable  BCS  format  standards  for  each 
document.  Documents  are  typed  using  ATMS  (Advanced  Text  Management 
System  -  IBM)  techniques  according  to  specified  format  standards.  No  formal 
auditing  of  adherence  to  these  format  standards  is  performed.  However, 
informal  auditing  occurs  by  the  ATMS  typists  (who  are  familiar  with  the 
standards),  and  several  IPAD  managers  (Program  Support/Quality  Assurance 
Manager,  Program  Manager,  the  manager  responsible  for  document  develop¬ 
ment,  etc.).  Formal  and  informal  control  of  document  adherence  to  standards 
can  provide  performance  indicators  by  the  number  of  errors  detected  and 
corrected.  This  information  can  be  obtained  via  the  Quality  Assurance  Check 
Sheet  (see  Figure  6.3-3). 

11.  Software  development  tools  in  the  area  of  design  analysis  and  verification 
provide  documentation  of  the  existing  design  data  base,  thus  supplying  reports 
indicating  the  status  of  the  design  effort.  The  system  currently  produces  nine 
reports.  An  example  of  such  a  report  provided  by  the  Integrated  Design 
Analysis  Programs  (IDAP)  system  is  shown  in  Figure  6.3-4.  This  report,  called 
the  Full  PDL  Report,  provides  current  information  about  components  of  the 
design,  variables  referenced  by  design  modules,  and  the  state  of  design 
decomposition.  Reports  which  have  been  generated  earlier  in  the  project 
effort  can  be  manually  compared  with  current  reports  to  provide  an  indication 
of  design  progress.  The  IDAP  system  has  a  predefined  set  of  constructs  which 
can  be  used  to  represent  a  design.  The  use  of  a  construct  not  part  of  the 
predefined  set  will  not  be  accepted  by  IDAP.  While  the  IDAP  system  does  not 
provide  indicators  of  performance,  use  of  the  system  must  conform  to 
standards;  therefore  a  degree  of  technical  excellence  is  implied. 

It  should  be  noted  that  the  IDAP  system  was  under  development  through  IPAD 
funding  until  November,  197S.  Due  to  time  and  money  constraints,  only  the 
first  phase  of  the  system  was  completed  when  IPAD-supported  development 
was  discontinued.  Therefore,  while  the  first  phase  is  available  for  use,  the 
system  is  not  highly  supported  by  IPAD  managers  or  designers.  Design  analysis 
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Figure  6.3-4  SAMPLE  I  DAP  FULL  PDL  REPORT  (Page  1) 


Figure  6.3-4  SAMPLE  I  DAP  FULL  PDL  REPORT  (Page  2) 


capabilities  are  currently  being  developed  by  an  IPAD-independent  group  and 
probably  will  not  be  available  for  timely  use  on  IPAD. 

12.  Software  configuration  management  provides  evidence  of  status  and  progress 
since  the  main  thrust  of  the  technique  is  to  maintain  the  integrity  of  all 
program  deliverables  and  other  key  program  documentation  and  software.  The 
configuration  control  process  consists  of  the  functions  of  status  accounting 
and  change  control.  The  basic  major  unit  of  work  to  be  controlled  is  a 
Configuration  Item  (Cl).  Usually  a  Cl  is  a  specific  document  or  unit  of 
software.  The  Cl  is  identified  and  defined  and  all  action  toward  completion  of 
work  on  the  Cl  is  documented  and  controlled.  Thus,  the  status  and  progress  of 
a  Cl  can  easily  be  determined.  Figure  6.3-5  provides  an  example  of  the 
configuration  control  process  for  document  development.  Each  document  goes 
through  50  steps  after  the  initial  writing  has  been  accomplished.  The  first  17 
steps  are  performed  on  the  review  copy  of  the  document,  while  steps  18  to  36 
are  performed  on  the  approval  copy  and  the  last  14  steps  (37-50)  apply  to  the 
final  copy.  Status  of  the  document  can  easily  be  determined  by  scanning  the 
"COMPLETED  BY"  column  of  the  Documentation  Process  Check  Sheet.  In  a 
like  manner,  progress  can  be  determined  by  the  progression  of  a  document 
through  this  cycle. 

The  Software  Problem  Report  form  (also  called  a  Discrepancy  Report)  is  used 
to  report  detailed  information  about  the  identification,  analysis  and  solution  of 
software  problems.  Figure  6.3-6  provides  a  sample  Software  Problem  Report 
form.  Initially  the  form  contains  a  description  of  the  problem  or  its 
symptoms.  As  the  problem  is  analyzed  and  corrected  the  form  is  used  to 
record  detailed  information. 

13.  The  requirements  specification  baseline,  after  formal  customer  approval,  is  in 
itself  an  indicator  of  status  (completion  of  the  requirements  phase)  and 
progress  (have  progressed  from  requirements  and  are  ready  to  begin 
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Figure  6.3-5  SAMPLE  DOCUMENTATION  PROCESS  CHECK  SHEET  (Page  1) 
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Figure  6.3-5  SAMPLE  DOCUMENTATION  PROCESS  CHECK  SHEET  (Page  2) 
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preliminary  design).  Customer  approval  of  the  baseline  is  an  indicator  of 
performance  during  the  requirements  task. 

During  the  requirements  analysis  task  each  requirement  is  analyzed  for 
compliance  with  specified  criteria  (see  Section  17,  Status  Reviews,  for  a 
complete  description  of  the  criteria  during  a  requirements  review).  Also, 
during  this  task  the  user  requirements  are  refined  and  the  requirements 
baseline  is  developed.  Requirements  analysis  is  performed  prior  to  the 
Requirements  Review  with  the  customer.  During  this  analysis,  a  form  is 
completed  for  each  requirement.  The  first  page  of  the  Requirements  Analysis 
Form,  shown  in  Figure  6.3-7,  describes  the  requirement,  its  source,  acceptance 
criteria,  and  approval  signatures  of  the  leader  of  the  requirements  definition 
team  and  the  leader  of  the  requirements  analysis  team.  The  second  page  of 
this  form  (see  Figure  6.3-8)  contains  a  checklist  of  the  ten  attributes  of  a 
requirement,  questions  to  be  answered  by  the  requirements  definition  team, 
recommendations  from  the  requirements  analysis  team  and  a  statement  about 
the  feasibility  of  satisfying  the  requirement.  Key  IPAD  individuals  can  easily 
ascertain  the  status  of  each  requirement  during  the  analysis  effort  and  are 
able  to  monitor  analysis  progress  by  studying  the  Report.  In  addition,  the 
performance  of  the  requirements  definition  group  can  be  indicated  via  the 
attribute  checklist. 

14.  Technical,  schedule  and  cost  reports  form  the  basis  of  status  and  progress 
evaluation  for  cost  and  schedule  control. 

A.  533-M  (monthly)  Report 

The  information  is  broken  down  to  WBS  level  2  and  is  also  summarized  at  level 
1  and  for  the  total  project.  Cost  data  is  broken  down  by  cost  element  and 
computing  labor  is  separated  from  engineering  and  manufacturing  labor.  The 
major  cost  elements  are:  3  labor  categories,  material,  computer  cost,  travel, 
5  overhead  and  fringe  benefit  categories,  and  facilities  cost.  The  report 
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REQUIREMENTS  ANALYSIS  FORM 
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Figure  6.3-7  SAMPLE  REQUIREMENTS  ANALYSIS  FORM 
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REQUIREMENTS  ANALYSIS  CRITERIA  CHECKLIST 
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6.3-8  SAMPLE  REQUIREMENTS  ANALYSIS  CRITERIA  CHECKLIST 


contains  the  number  of  hours  expended  in  the  various  labor  categories  and  the 
computer  cost  units  expended.  Figures  6.3-9(l),  6. 3-9(2),  and  6. 3-9(3)  illus¬ 
trate  samples  of  the  533-M  report  at  WBS  level  2,  level  1,  and  for  the  total 
program,  respectively.  This  report  is  also  produced  quarterly  (533-Q).  The 
monthly  and  quarterly  reports  have  a  standard  content  which  is  fixed  by  NASA 
and  is  part  of  IPAD's  reporting  requirement  to  NASA.  From  the  reports,  cost 
data  can  be  extracted,  manipulated  and  evaluated.  The  status  of  WBS  level  1 
and  2  tasks  can  easily  be  obtained.  Comparison,  of  current  to  past  reports 
evidences  progress.  Performance  can  be  partially  viewed  by  comparison  of 
budgeted  to  actual  costs.  The  IPAD  Business  Manager  uses  data  from  the  533 
reports  to  analyze  the  variance  between  actual  and  budgeted  expenses.  While 
it  may  seem  a  straightforward  process  of  subtraction,  this  is  not  the  case.  A 
variance  may  be  caused  by  many  factors.  A  few  examples  are: 

•  a  schedule  slippage 

•  a  labor  rate  change 

•  an  overhead  rate  change 

•  overtime  hours  expended 

•  extra  labor  hours  expended 

•  replanning  causing  rescheduling  (i.e.,  execution  of  a  task  before  or 
after  its  scheduled  execution) 

After  variance  analysis  is  complete,  clear  evidence  of  the  program's  progress 
and  performance  in  relation  to  plan  is  available.  Variance  analysis  presents 
the  necessary  indicators  to  explain  why  progress  has  varied  from  plan. 

B.  IPAD  Integrated  Resource/Schedule  Status  Report 

This  report  summarizes  schedule  information  on  the  upper  portion  of  the  page 
and  resource  data  on  the  lower  portion.  Special  symbols  are  used  to  portray 
milestones  (contract  and  non-contract),  schedule  slides,  and  estimated 
completion  dates  (ahead  or  behind  schedule).  Planned  and  actual  hour 
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Figure  6. 3-9(1)  SAMPLE  533-M  REPORT  (WBS  LEVEL  2) 


Figure  6. 3-9(2)  SAMPLE  533-M  REPORT  (WBS  LEVEL  1) 


Figure  6. 3-9(3)  SAMPLE  533-M  REPORT  (TOTAL  PROGRAM) 


expenditures  are  graphically  and  numerically  portrayed  for  easy  comparison. 
A  sample  Integrated  Resource/Schedule  Status  report  is  illustrated  in  Figure 
6.3-10  for  WBS  task  1.7  (level  2  task).  Status  and  progress  are  concisely  shown 
and  performance  in  relation  to  plan  is  highly  visible. 

C.  Work  Schedule  Status  Report 

This  report  is  produced  weekly  and  provides  indicators  of  status  and  progress. 
It  is  a  computerized  report  which  uses  symbols  to  indicate  task  start,  end,  and 
intervening  work.  It  is  produced  internally  at  WBS  levels  4  and  3  and  is 
summarized  at  level  2.  Figure  6.3-11  illustrates  the  Work  Schedule  Status 
report  at  level  3,  and  summarized  at  level  2  for  the  period  from  January  1978 
through  September  1978.  Notice  that  the  four  columns  at  the  far  right  of  the 
report  separate  manhours  expended  into  engineering  and  computer  program¬ 
ming  categories.  At  completion,  the  budgeted  and  estimated  manhours 
expended  are  provided. 

This  report  provides  quick  status  and  progress  visibility  on  a  weekly  basis  and 
therefore  is  an  invaluable  management  tool. 

13.  The  findings  of  an  independent  test  organization  provide  evidence  of  perform¬ 
ance.  Independent  testing  involves  analysis  of  software  requirements,  admin¬ 
istration  and  execution  of  tests,  review  of  test  results,  inconsistency 
reporting  and  retesting  as  necessary  until  all  requirements  have  been  met. 
Independence  of  the  test  group  from  the  software  developers  will  introduce  an 
unbiased  evaluation  of  the  capabilities  and  provide  an  indication  of  the 
performance  of  the  software. 

16.  Checkpoint  reviews  are  used  to  evaluate  project  activities.  Performance  is 
indicated  by: 

-  evaluation  of  project  progress  against  schedule 

-  resource  usage  evaluation 
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-10  SAMPLE  INTEGRATED  RESOURCE/SCHEDULE  REPORT 


Figure  6.3-11  SAMPLE  WORK  SCHEDULE  STATUS  REPORT 


-  problem  identification  and  correction 


17. 


Status  reviews,  conducted  by  key  individuals  to  review  documentation,  deliver¬ 
ables  and  project  achievement,  provide  evidence  of  status  and  progress. 
Criteria  which  apply  to  particular  reviews  are  normally  specified  prior  to  the 
review.  For  example,  criteria  which  apply  to  the  Requirements  Document 
Review  are: 

A.  Complete 

All  items  that  are  needed  for  the  specification  of  the  requirements  of 
the  solution  to  the  problem  have  been  included. 

B.  Correct 

Each  item  in  the  requirements  specification  is  free  from  error. 

C.  Precise,  unambiguous  and  clear 

Each  item  is  exact  and  not  vague,  there  is  a  single  interpretation  of  each 
item,  the  meaning  of  each  item  is  understood  and  the  specification  is 
easy  to  read. 

D.  Consistent 

No  item  conflicts  with  another  item. 

E.  Relevant 

Each  item  is  pertinent  to  the  problem  and  its  solution. 

F.  Testable 

During  program  development  and  acceptance  testing,  it  will  be  possible 
to  determine  whether  the  item  has  been  satisfied. 

G.  Traceable 

Each  item  can  be  traced  to  its  origin  in  the  problem  environment. 
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H.  Feasible 

Each  item  can  be  implemented  with  the  techniques,  tools,  resources,  and 
personnel  that  are  available  within  the  specified  cost  and  schedule  con¬ 
straints. 

I.  Free  of  Unwarranted  Design  Detail 

The  requirements  specifications  are  a  statement  of  the  requirements 
that  must  be  satisfied  by  the  problem  solution  and  they  are  not  obscured 
by  proposed  solutions  to  the  problem. 

J.  Manageable 

The  requirements  specifications  are  expressed  in  such  a  way  that  each 
item  can  be  changed  without  excessive  impact  on  another  item.  Changes 
to  the  completed  specifications  can  be  controlled.  Each  proposed  change 
can  be  traced  to  an  existing  requirement  and  the  impact  of  the  proposed 
change  can  be  assessed. 

Achievement  of  each  of  these  criteria  provides  indicators  of  performance  for 
the  specification  and  documentation  of  requirements. 

18.  A  complete  preliminary  design  is  evidenced  by  many  factors.  A  sample  of 
these  factors  is  given  in  a  draft  of  the  Software  Development  Procedure  for  a 
Preliminary  Design  Review  for  IPAD.  A  draft  PDR  document  review  checklist 
has  been  developed  by  IPAD.  A  summary  of  this  checklist  follows: 

A.  Are  the  PD  objectives  clearly  stated? 

B.  Does  the  PD  document  contain  a  description  of  the  procedure  used  to  do 
PD  or  is  there  a  reference  to  such  a  procedure? 

C.  Is  there  a  list  of  functions  to  be  provided  by  the  computing  system? 

D.  Is  there  a  model  of  the  user  interface  to  the  computing  system? 

E.  Are  there  models  and/or  descriptions  of  all  other  interfaces  to  the 
computing  system? 
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F.  Is  there  a  high-level  functional  model  of  the  proposed  computing  system? 

G.  Are  the  major  implementation  alternatives  and  their  evaluations  pre¬ 
sented  in  the  document? 

H.  Is  there  a  recommendation  from  the  PD  team  to  implement  one  of  the 
alternatives? 

I.  Is  the  recommendation  adequately  supported? 

3.  Does  the  information  presented  in  the  PD  document  and  during  PDR  give 
confidence  that  the  computing  system  can  be  implemented  to  satisfy  the 
requirements  to  such  an  extent  that  you  would  use  the  system? 

19.  Project  Control/70  (a  proprietary  product  of  Atlantic  Software,  Inc.)  is  a 
software  package  used  for  project  planning,  reporting,  analysis  and  accounting. 
The  Project  Status  and  Executive  Summary  Reports  provide  an  accurate 
picture  of  project  status  and  the  current  progress  on  a  task  basis.  The  Active 
Projects  Due  and  Over  Budget  Reports  highlight  which  tasks  need  special 
attention  due  to  poor  schedule  or  cost  performance. 

6 .4  INDICATOR  RESPONSIBILITY 

Each  of  the  indicators  described  earlier  as  being  used  by  IPAD  to  assess  status, 
progress  and  performance  is  the  formal  responsibility  of  one  or  more  IPAD  team 
member.  These  team  members  have  been  formally  delegated  the  responsibility  for 
establishment,  control  and  maintenance  of  the  indicators.  Other  team  members 
have  been  formally  directed  to  interpret  program  status,  progress  and  performance 
indicators  in  specific  areas.  Figure  6.4-1  illustrates  formally  directed  responsi¬ 
bilities  for  control  and  interpretation  of  the  indicators. 

It  is  important  to  remember  that  the  many  informal  relationships  between  indicators 
and  responsibilities  have  not  been  incorporated  in  Figure  6.4-1,  since  they  are  ill- 
defined.  The  matrix  illustrates  major  responsibilities  for  indicators.  Often  an 
individual  with  responsibilities  must  have  approval  or  input  from  one  or  more  other 
IPAD  individuals  before  his  work  will  be  consolidated  into  the  IPAD  framework. 
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This  approval  relationship  is  not  depicted  in  the  figure.  Also,  the  matrix  may  depict 
an  individual  as  being  responsible  for  an  indicator  when  the  individual  may  delegate 
responsibility  to  his  staff. 

Figure  6.4-1  does  not  list  the  IPAD  Program  Manager  (PM)  as  one  of  the  responsible 
individuals.  The  PM  has  overall  authority  to  control  and  establish  each  indicator. 
He  uses  data  provided  by  each  indicator  to  assess  program  status,  progress  and 
performance. 

In  a  similar  fashion,  the  NASA  IPAD  Project  Office  has  an  overall 
coordinating/monitoring  function.  They  review  and  approve  program  products  and 
use  the  data  obtained  from  the  indicators  (which  has  been  tabulated  or  summarized 
by  IPAD)  to  assess  progress,  status  and  performance. 

Additionally,  various  Boeing  groups  (i.e.,  Finance  and  Contracts)  provide  support  in 
the  areas  of  technical  involvement,  reviews,  and  technical,  schedule  and  cost 
reports. 

The  Project  Control/70  (PC/70)  system  provides  reports  to  all  individuals  about 
resource  planning  and  consumption  as  well  as  budget  and  expenditures  for  particular 
work  unit  tasks.  Input  to  PC/70  comes  from  all  of  the  IPAD  team  leaders  and 
managers  in  the  form  of  Work  Authorizations  (see  Section  2. 2. 1.9)  or  via  IPAD  Task 
Resource  Forms. 

6.5  IND1CATOR/OB3ECT1VE  RELATIONSHIP 

Each  of  the  indicators  which  IPAD  is  using  to  assess  status,  progress  and  perform¬ 
ance  has  been  applied  in  order  to  support  achievement  of  one  or  more  of  the 
program  objectives.  Since  the  relationship  between  program  objectives  and  manage¬ 
ment  techniques  has  been  surveyed  (see  Section  4.3),  the  relationship  between 
indicators  provided  by  the  techniques  and  the  program  objectives  can  be  inferred  by 
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the  results  of  the  survey.  Figure  6.5-1  illustrates  the  derived  relationship  between 
indicators  and  objectives. 

When  two  techniques  have  been  grouped  together  as  one  indicator,  the  surveyed 
responses  have  been  combined  also.  For  example,  the  techniques  of  User  Involve¬ 
ment  Planning  and  Industry  and  Technical  Involvement  have  been  grouped  under  the 
category  of  user,  industry  and  technical  involvement  indicators.  As  Appendix  E 
shows,  both  techniques  were  surveyed  to  be  strongly  supportive  of  the  objective 
"Usefulness  Demonstration."  Therefore  the  combined  group  also  has  a  strong 
relationship.  When  the  surveyed  relationship  differs  between  the  grouped  tech¬ 
niques,  the  combined  relationship  must  reflect  this  also.  For  example,  User 
Involvement  Planning  was  surveyed  to  be  strongly  related  to  the  User  Involvement 
objective,  while  Industry  and  Technical  Involvement  is  only  moderately  related  to 
the  same  objective.  The  combined  relationship  is  strongly  moderate,  recorded  as  M+ 
in  Figure  6.5-1.  Other  combined  relationships  have  been  similarly  evaluated  and 
recorded. 

Since  Project  Control/70  was  not  surveyed  along  with  the  original  candidate 
techniques,  a  strong  or  moderate  relationship  cannot  be  assessed  in  the  same  manner 
as  other  indicators.  However,  from  conversations  with  IPAD  program  management 
and  study  of  PC/70  literature,  it  has  been  determined  that  PC/70  is  being  used  to 
achieve  program  objectives  of  cost,  schedule,  and  visibility.  Furthermore,  the 
system  is  having  widespread  usage  on  IPAD,  indicating  that  it  strongly  supports 
achievement  of  these  objectives. 

From  study  of  Figure  6.5-1,  a  few  observations  may  be  made.  First,  seven  of  the 
objectives  have  a  strong  or  strong-moderate  relationship  to  at  least  one  of  the 
indicators.  Of  the  six  remaining  objectives,  4  have  6-9  indicators  which  moderately 
relate  to  them. 

The  last  two  objectives,  machine  independence  and  contractor  commitment,  do  not 
have  x  sufficiently  observable  relationship  to  any  of  the  indicators.  However, 


INDICATORS 


Relationship  determined  via  research,  not  through  survey. 

Figure  6.5-1  RELATIONSHIP  BETWEEN  INDICATORS  AND  OBJECTIVES 


neither  of  these  objectives  fall  into  the  category  of  the  three  highest-ranked 
objectives  (see  Section  4.1).  Therefore,  it  is  not  surprising  to  find  that  they  are  not 
observably  supported  by  indicators.  Also,  contractor  commitment  is  more  intangible 
than  the  rest  of  the  objectives,  therefore,  less  measurable. 

The  lack  of  indicators  providing  visibility  into  the  objective  of  machine  indepen¬ 
dence  deserves  closer  scrutiny. 

Generally,  machine  independence  is  supported  by  the  adoption  and  use  of  program¬ 
ming  standards.  Explicit  conformance  to  ANSI  Standards  will  assure  that  no  unique 
(machine  dependent)  features  are  designed  into  the  system.  However,  only  four 
survey  participants  indicated  a  strong  or  moderate  relationship  between  standards 
and  machine  independence.  This  could  be  due  to  the  fact  that  little  formal  auditing 
to  standards  is  performed  on  IPAD.  The  participants  may  have  felt  that  the 
informal  auditing  to  be  performed  would  not  contribute  significantly  to  machine 
independence. 

6.6  MANAGEMENT  TECHNIQUE  INDICATORS 

Each  of  the  techniques  used  by  IPAD  was  initiated  to  support  achievement  of  an 
objective  or  set  of  objectives.  As  a  whole,  the  techniques  were  designed  and 
incorporated  to  provide  status,  progress  and  performance  indicators  (SPP  indi¬ 
cators).  While  application  of  some  of  the  techniques  is  more  abstract  than  others, 
there  exists  (within  IPAD)  evidence  of  the  use  of  the  techniques. 

Each  of  the  techniques  being  used  by  IPAD  was  studied  to  determine  ways  in  which 
the  technique  application  could  be  evaluated.  That  is,  management  technique 
indicators  (hereafter  refered  to  as  MT  indicators)  which  would  provide  an  unbiased 
evaluation  of  the  "goodness"  of  technique  application  were  identified.  These  MT 
indicators  were  in  the  form  of  data  to  be  collected  and  critical  questions  which 
would  determine  whether  the  technique  was  providing  any  function  not  otherwise 
provided. 
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The  list  which  follows  identifies  data  which  should  be  provided  or  critical  functions 
which  would  be  furnished  by  the  use  of  a  particular  technique.  The  list  is  not 
necessarily  a  complete  list  of  every  MT  indicator  which  would  assess  the  application 
of  each  technique.  It  is,  however,  an  unbiased  list  of  generally-recognized  data  or 
functions  which  would  allow  evaluation  of  management  techniques, 

A.  Project  Manager  Concept 

1.  Is  there  explicit  use  and  delegation  of  authority? 

2.  Is  there  explicit  definition  of  responsibility? 

B.  Work  Breakdown  Structure/Schedule 

1.  Is  there  any  overlapping  or  redundancy  of  project  tasks? 

2.  How  much  difficulty  is  there  in  obtaining  the  overall  status  of  the 
project  by  determining  the  status  of  elements  which  make  up  the  overall 
structure? 

3.  Are  members  of  individual  work  units  aware  of  how  their  units  fit  into 
the  overall  structure? 

C.  Programmers'  Handbook 

1.  Are  the  requirements/specifications  for  this  project  centralized  and 
easily  ascertained  by  a  programmer? 

2.  How  long  does  it  take  to  determine  the  tools  and  techniques  available  to 
the  designer/programmer? 

3.  How  long  does  it  take  to  assemble  the  project  development  test 
guidelines? 

D.  Project  File 

1.  How  hard  is  it  to  find  a  particular  item  of  correspondence  about  the 
project? 

2.  How  long  does  it  take  to  find  out  action  items  that  have  resulted  from  a 
particular  review? 
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3.  How  long  does  it  take  to  determine  what  changes  have  been  made  to  the 
project  SOW? 

E.  Software  Notebook 

1.  How  long  does  it  take  to  determine  the  status  of  a  particular  module? 

2.  What  effort  has  gone  into  code  development  and  how  long  does  it  take  to 
determine  this  information? 

3.  How  long  does  it  take  to  determine  test  case  results  for  each  module? 

F.  Support  Library 

1.  How  hard  is  it  to  obtain  a  copy  of  the  official  version  of  a  program? 

2.  Can  a  history  of  controlled  program  changes  be  obtained? 

3.  How  long  does  it  take  to  obtain  a  list  of  the  3CL  needed  to  execute  a 
program? 

G.  Work  Authorization  Form 

1.  How  hard  is  it  to  find  out  when  an  item  is  scheduled  for  completion  and 
what  the  total  cost  is  estimated  to  be? 

2.  How  does  the  cost  of  each  item  contribute  to  the  total  cost  of  the 
system? 

3.  Is  there  any  work  being  done  which  is  not  specified  or  supported  by  Work 
Authorizations? 

H.  Project  Control  Room 

1.  How  long  does  it  take  to  determine  the  status  of  program  cost  and 
schedule? 

2.  How  long  does  it  fake  to  obtain  a  conference  room  for  a  project 
meeting? 

I.  Walk-through/Inspections 

1.  How  well  do  other  programming  group  members  understand  the  module 
logic  and  data  flow  of  other  members'  code? 
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Is  there  a  record  of  errors,  discrepancies,  and  inconsistencies  discovered 
during  programming  group  reviews? 

3.  Does  the  code  of  the  various  members  of  the  programmer  team  interface 
smoothly  (or  how  many  interface  errors  were  detected)? 

4.  Do  work  products  receive  a  formal  approval  or  rejection  as  a  result  of 
inspections/walk-through? 

3.  Quality  Assurance  Reviews  and  Audits 

1.  Are  quality  assurance  people  aware  of  project  status? 

2.  Do  explicit  measures  of  quality  assurance  exist? 

3.  Are  quality  assurance  reports  produced  which  document  audit  results? 

K.  Resource  Allocation  Sheets/Resource  Requirements  Summary 

1.  Is  there  consistent  manloading  of  the  project  resources? 

2.  Can  the  job  of  an  individual  be  determined  by  documentation  at  any 
point  in  time? 

3.  Have  conflicts  resulted  due  to  lack  of  proper  facilities  at  the  needed 
time? 

L.  Programming  Standards 

1.  Is  there  a  standards  manual  or  written  programming  instructions? 

2.  Are  audits  or  tests  performed  to  determine  compliance  to  written 
standards? 

3.  Are  records  kept  documenting  non-standard  programming  practices  used 
and  detected? 

M.  Incremental  Development 

1.  Are  design  component  descriptions  written  as  the  last  component  pro¬ 
duced  plus  a  new  capability? 

2.  Is  any  given  design  component  independently  testable? 
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N.  User  Involvement  Planning/Industry  and  Technical  Involvement 

1.  Do  user,  industry  and  technical  people  attend  major  component/project 
reviews? 

2.  Does  documentation  of  user,  industry  and  technical  feedback,  opinions 
and  recommendations  exist? 

3.  Is  there  a  specific  plan  to  involve  these  people  in  reviews  and  feedback? 

O.  Documentation  Standards 

1.  Are  there  formal  standards  concerning  the  content  and  format  of 
documentation? 

2.  Are  the  documents  formally  audited  for  compliance  with  content  and 
format  standards? 

P.  Design  Representation  and  Testing  Tools 

1.  Has  a  design  representation  scheme  been  adopted  by  the  project? 

2.  Do  programs  exist  which  perform  design  analysis  and  consistency  verifi¬ 
cation? 

3.  Is  design  documentation  produced  by  a  tool  which  would  otherwise  not 
exist? 

Q.  Complete  Preliminary  Design 

1.  Is  there  a  Preliminary  Design  review  conducted  which  checks  PD 
completeness? 

2.  Are  detailed  design  analysis  results  available? 

3.  How  many  sections  of  the  PD  have  been  specified  as  "to  be  completed" 

later? 

R.  Configuration  Control 

1.  Are  specific  forms  being  used  to  identify,  account  for,  control  and  verify 
changes  to  the  project? 

2.  How  long  does  it  take  to  determine  the  current  configuration? 

3.  How  long  does  it  take  to  effect  a  system  change? 
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Requirements  Specification  Baseline 

1.  Is  there  customer  approval  of  the  requirements  specification? 

2.  Is  there  a  requirements  document  which  follows  established  guidelines? 

3.  Do  the  requirements  map  back  to  a  Software  Specification  or  the 
Statement  of  Work? 

Technical,  Schedule  and  Cost  Reports 

1.  Are  technical,  schedule  and  cost  reports  produced  or  is  there  a  plan  to 
produce  them? 

2.  Is  there  a  central  collection  organization  which  serves  to  collect  data  for 
these  reports? 

3.  Is  there  some  analysis  of  the  content  of  these  reports  which  shows  that 
they  are  being  used? 

it.  Is  there  cost,  schedule  or  technical  data  available  which  would  not 
otherwise  have  been  produced? 

Independent  Test  Evaluation 

1.  Is  there  an  independent  test  group  which  is  organizationally  separate 
from  the  project  team? 

2.  Is  there  a  test  plan  which  is  derived  from  the  requirements  or  which  is 
derived  independent  of  development  and  design  effort? 

3.  Is  there  a  test  plan  in  existence  prior  to  completion  of  design  and 
development? 

Project  Checkpoints 

1.  Is  there  a  chart  which  illustrates  major  project  events  during  the  project 
time  frame? 

2.  Are  major  project  deliverables  identified? 

Checkpoint  Reviews 

1.  Is  there  a  series  of  reviews  which  are  associated  to  project  checkpoints? 
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Is  there  some  plan  which  specifies  what  reviews  will  be  held  and  ties 
them  to  project  deliverables  or  products? 

3.  Are  there  reports  which  were  generated  as  a  result  of  program  product 
reviews? 

X.  Status  Reviews 

1.  Are  there  project  reviews  which  are  held  at  regular  intervals  (i.e., 
monthly)  or  periodically  due  to  a  requirement  to  assess  project  status? 

2.  Are  reports  generated  periodically  which  assess  project  status? 

6.7  ADDITIONAL  MANAGEMENT  TECHNIQUE  INDICATORS 

The  list  of  generally-recognized  products  or  indicators  of  effective  use  of  manage¬ 
ment  techniques  (see  Section  6.6)  was  studied  in  relation  to  IPAD.  If  the  products  or 
indicators  identified  did  not  appear  to  be  provided  through  the  use  of  an  IPAD 
technique,  the  need  for  an  additional  indicator  of  technique  effectiveness  was 
identified.  In  other  words,  if  the  answer  to  each  of  the  preceding  questions  could  be 
presently  determined,  no  additional  indicators  would  be  required.  The  questions 
which  could  not  be  answered  suggest  that  additional  indicators  are  needed  to  assess 
the  application  of  the  techniques. 

The  identification  of  additional  MT  indicators  needed  does  not  imply  a  lack  on  the 
part  of  IPAD,  since  IPAD  has  not  been  commissioned  to  evaluate  technique 
effectiveness.  It  does  imply  that  more  data  would  have  to  be  collected  than  is 
presently  available  on  the  IPAD  project. 

The  following  additional  MT  indicators  would  be  required  for  an  evaluation  of  the 
application  of  management  techniques: 

1.  Data  would  need  to  be  collected  to  explicitly  show  the  use  and  delegation  of 
authority  (refer  to  Question  No.  1,  Project  Manager  Concept).  One  way  this 
might  be  illustrated  would  be  to  use  the  Management  Responsibility  Guide 


(MRG).  The  MRG  presents  an  approach  to  clarify  the  role  each  manager  plays 
in  relation  to  his  work  group  and  to  the  organization. 

Some  means  would  have  to  be  developed  to  determine  whether  facilities  were 
actually  available  when  needed.  This  might  be  determined  by  talking  with 
people  to  assess  their  satisfaction/dissatisfaction  with  facilities  availability 
since  no  data  exists  which  could  be  tabulated.  (Refer  to  Question  No.  3, 
Resource  Allocation  Sheets/PC/70.) 

Some  means  for  auditing  of  conformance  to  programming  standards  would 
have  to  be  developed.  (Refer  to  Question  No.  1,  Programming  Standards.) 
Tools  are  in  existence  within  the  industry  which  document  conformance  to 
ANSI  standards  (i.e.,  PFORT  verifier  put  out  by  Bell  Labs)  and  to  generally- 
accepted  programming  standards  (i.e.,  TRW's  Code  Auditor).  Specialized 
standards,  however,  would  have  to  be  manually  audited.  Some  analysis  of 
conformance  to  standards  is  performed  during  the  design  phases  of  IPAD.  The 
IDAP  tool,  if  used,  would  prohibit  some  non-standard  constructs,  and  others 
may  be  pointed  out  during  structured  walk-throughs.  However,  no  formal 
auditing  of  program  code  is  performed. 

In  line  with  No.  3  above,  no  records  are  kept  documenting  non-standard 
programming  practices  detected.  (Refer  to  Question  No.  3,  Programming 
Standards.)  For  example,  no  data  showing  how  many  non-standard  lines  of 
code  exist  is  recorded.  Output  from  tools  like  those  referenced  in  No.  3  could 
be  tabulated,  if  they  were  used. 

While  documents  are  formally  audited  for  conformance  to  content  standards, 
no  auditing  of  format  standard  conformance  is  performed.  (Refer  to  Question 
No.  2,  Documentation  Standards.)  This  auditing  would  probably  have  to  be 
performed  manually  and  results  tabulated. 


A  test  group  which  is  independent  of  the  IPAD  project  does  not  exist  and 
would,  therefore,  not  be  available  for  assessment  purposes.  However,  the 
Configuration  Control  group  (which  is  separate  from  actual  development  of 
IPAD  software)  is  responsible  for  development  and  acceptance  testing.  Since 
the  test  group  is  not  organizationally  separate  from  IPAD,  it  is  possible  that 
sufficient  indicators  of  the  independent  testing  technique  might  not  be 


available. 


7.0  GENERALIZED  MANAGEMENT  TOOL  MODEL 


7 . 1  SELECTION  OF  TECHNIQUES  AND  INDICATORS 


Management  techniques  are  selected  to  support  the  program  objectives  (see  Appen¬ 
dix  E).  Indicators  of  status,  progress  and  performance  are  chosen  to  provide 
visibility  into  the  achievement  of  program  objectives  (see  Figure  6.5-1).  Illustrated 
pictorially,  these  relationships  and  dependencies  can  be  modeled  as  follows: 


Program  objectives  are  generally  derived  jointly  during  meetings  between  the 
Program  Manager  and  the  customer.  SPP  indicators  are  maintained  and  used  by  key 
individuals  to  monitor  project  development  (see  Figure  6.4-1).  The  development 
team  is  generally  supported  by  management  techniques  during  the  project  effort.  If 
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the  application  of  the  techniques  is  to  be  evaluated,  MT  indicators  are  used. 
Incorporating  elements  of  key  responsibilities  (R)  and  MT  indicators  into  the  model 
previously  illustrated,  Figure  7.1-1  results: 


Figure  7.1-1  GENERALIZED  MANAGEMENT  TOOL  MODEL 

During  earlier  research,  it  became  apparent  that  the  5PP  indicators  were  provided  by 
the  use  of  management  techniques.  Due  to  this  observed  relationship,  it  seems 
reasonable  that  the  selection  of  SPP  indicators  and  management  techniques  should  be 
more  closely  associated.  The  selection  of  techniques  should  be  based  upon  the  SPP 
indicators  chosen  as  well  as  on  the  program  objectives. 
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Selection  of  techniques  should  begin  by  formulation  of  a  set  of  program  objectives. 
Next,  those  indicators  which  would  provide  visibility  of  the  achievement  of  the 
objectives  should  be  selected.  Since  the  indicators  are  generally  obtained  through 
the  implementation  of  management  techniques,  an  initial  set  of  techniques  has  also 
been  determined.  This  set  should  be  extended  by  inclusion  of  those  techniques  which 
directly  support  the  objectives  (see  Appendix  E). 

To  monitor  the  techniques,  evidence  of  their  application  should  be  kept.  This 
evidence,  provided  through  MT  indicators,  should  act  as  an  early  warning  device  of 
potential  development  problems. 

To  exemplify  the  use  of  the  model  illustrated  in  Figure  7.1-1,  several  scenarios  have 
been  developed  using  Appendix  E  and  Figures  6.4-1  and  6.5-1. 

Scenario  A  Data: 

The  customer  and  the  Program  Manager  have  compiled  a  list  of  program  objectives. 
One  objective  is  that  of  not  expending  more  than  the  estimated  cost  of  the 
development  effort.  Use  of  the  matrix  presented  in  Figure  6.5-1  shows  that 
achievement  of  this  objective  is  made  visible  through  cost  reports,  checkpoint 
reviews  of  cost  data,  and  use  of  Project  Control/70  resource  reports.  These 
indicators  are  provided  through  the  techniques  of  technical,  schedule  and  cost 
reports,  checkpoint  reviews  and  the  Project  Control/70  system.  From  Figure  6.4-1 
we  determine  that 

-  technical,  schedule  and  cost  reports  are  established  and  maintained  by  the 
Business  Manager  (with  the  finance  group  being  the  central  collection 
agency).  Furthermore,  the  reports  are  used  by  the  Business  Manager  (BM) 
and  the  Software  Development  Manager  (SDM)  to  assess  status,  progress 
and  performance. 


Resource  Requirements  Allocation  and  PC/70  are  established,  maintained 
and  used  by  all  the  managers  except  the  ITAB  Interface  Manager. 

The  objective  of  meeting  expected  costs  is  supported  by  the  following  techniques  as 
shown  in  Appendix  E: 

•  Project  Manager  Concept 

•  Work  Breakdown  Structure/Schedule 

•  Work  Authorization  Forms 

•  Resource  Allocation  Sheets/Project  Control/70 

•  Technical,  Schedule  and  Cost  Reports 

•  Project  Checkpoints 

•  Checkpoint  Reviews 

The  application  of  these  techniques  can  be  evaluated  through  the  use  of  MT 
indicators. 

1.  Project  Manager  Concept  MT  Indicators: 

•  Explicit  use  and  delegation  of  authority 

•  Explicit  definition  of  responsibility 

2.  Work  Breakdown  Structure  MT  Indicators: 

•  No  overlapping  or  redundant  tasks 

•  Visible  project  status  via  determination  of  status  of  components 

•  Individual  component  awareness  of  total  project  scope 

3.  Work  Authorization  MT  Indicators: 

•  Easy  access  to  estimated  completion  dates  and  costs 

•  Visibility  into  contribution  of  each  component  toward  total  system 

•  No  unauthorized  work  being  performed 


4. 


Resource  Allocation  MT  Indicators: 


•  Consistent  resource  loading 

•  Easy  determination  of  an  individual's  job  at  any  point  in  time 

•  No  conflicts  due  to  facility  availability 

5.  Technical,  Schedule  and  Cost  MT  Indicators: 

•  Reports  produced  or  a  plan  to  produce  them 

•  Centralized  data  collection  organization 

•  Analysis  of  reports  is  performed 

•  Data  produced  which  is  otherwise  unavailable 

6.  Project  Checkpoint  MT  Indicators: 

•  Major  project  events  identified  and  time  frame  for  their  occurrence 
documented 

•  Major  deliverables  identified 

7.  Checkpoint  Review  MT  Indicators: 

•  Series  of  reviews  associated  with  checkpoints 

•  Review  plan  in  existence 

•  Reports  produced  documenting  reviews 

Inserting  this  scenario  data  into  the  model,  Figure  7.1-2  results. 
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Figure  7.1-2  GENERALIZED  MODEL  OF  SCENARIO  A 
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Ideally,  the  MT  indicators  would  be  used  to  make  the  techniques  self-monitoring. 
For  example,  the  four  indicators  associated  with  technical,  schedule  and  cost 
reports  would  continuously  feed  back  information  to  the  Business  Manager  (BM)  and 
Software  Development  Manager  (SDM).  If  the  reports  were  not  produced,  the 
central  organization  not  formed,  analysis  was  not  performed  or  data  was  not 
available  the  key  individuals  would  know  that  the  technique  was  not  being  used 
effectively.  At  this  point  steps  would  be  taken  to  correct  the  situation. 

However,  most  problems  are  not  detected  until  the  SPP  indicators  are  viewed  by 
individuals  in  charge  of  project  assessment.  The  model  presented  will  illustrate  the 
resolution  of  problems  and  answering  of  questions  made  visible  by  the  SPP 
indicators. 

Using  the  data  from  Figure  7.1-2  it  is  possible  to  "walk-through"  the  model.  For 
example,  suppose  the  Business  Manager  receives  a  cost  variance  report  stating  that 
resource  expenditures  are  greater  than  planned.  He  knows  that  the  program 
objective  of  at  cost  is  not  being  met.  The  Program  Manager  is  informed  and  he 
determines  that  meeting  the  cost  objective  is  supported  by  the  following  techniques; 
Project  Manager  Concept,  Work  Breakdown  Structure,  Work  Authorization  Forms, 
Schedule  and  Cost  Reports,  Project  Checkpoints,  Checkpoint  Reviews,  and  Resource 
Allocation/Project  Control/70.  He  also  knows  which  individuals  are  concerned  with 
establishment  and  usage  of  those  techniques.  He  gathers  these  key  individuals  and 
tells  them  about  the  problem.  He  wants  to  know  what  caused  the  cost  overrun  and 
what  could  be  done  to  prevent  the  problem  in  the  future. 

Various  technical,  schedule  and  cost  reports  would  be  reviewed  by  the  BM  and  the 
SDM.  Reports  from  previous  checkpoint  reviews  would  be  gone  over  by  the  CD,  AD, 
TL  and  EDM  to  determine  whether  there  was  any  indication  that  a  cost  overrun 
would  occur  during  the  next  checkpoint  period.  The  Resource  Allocation  Sheets  and 
the  resource  reports  produced  by  the  Project  Control/70  system  would  be  studied  to 
determine  whether  more  resources  were  expended  than  planned.  One  of  the  MT 
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indicators  should  point  out  the  reason  for  the  cost  overrun  and  therefore  provide  a 
possible  means  of  correction  or  way  to  prevent  the  situation  from  recurring. 

The  overrun  could  have  been  caused  by  an  over  expenditure  of  resources,  or,  some 
facility  which  should  have  been  available  might  have  been  unavailable,  causing 
delays.  Or,  a  possible  overrun  might  have  been  indicated  in  the  previous  review 
report.  Another  cause  might  have  been  faulty  variance  computations,  or  erroneous 
charging  of  resource  expenditures. 

Another  scenario  was  developed  to  further  illustrate  the  studied  relationships.  Data 
is  presented  and  the  generalized  model  is  used  to  walk-through  the  scenario. 

Scenario  B  Data 

An  objective  developed  by  the  Program  Manager  and  the  customer  is  that  of 
machine  independence.  Although  not  observable  by  the  survey  performed  on  IPAD, 
this  objective  is  generally  made  visible  through  auditing  of  conformance  to  industry- 
recognized  standards  (i.e.,  ANSI  standards)  and  by  walk-throughs.  Programming 
standards  are  provided  through  the  technique  of  programming/ design  standards. 
Several  MT  indicators  provide  evidence  of  the  effective  application  of  this  tech¬ 
nique: 

1.  Is  there  a  standards  manual  or  written  programming  instructions? 

2.  Are  audits  or  tests  performed  to  determine  compliance  to  written 
standards? 

3.  Are  records  kept  documenting  non-standard  programming  practices  used 
and  detected? 

MT  indicators  providing  evidence  of  the  effective  application  of  structured  walk¬ 
throughs  are: 
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1.  How  well  do  other  programming  group  members  understand  the  module 
logic  and  data  flow  of  other  members  code? 

2.  Is  there  a  record  of  errors,  discrepancies  and  inconsistencies  discovered 
during  programming  group  reviews? 

3.  Does  the  code  of  the  various  members  of  the  programming  team 
interface  smoothly? 

4.  Do  work  products  receive  a  formal  approval  or  rejection  as  a  result  of 
walk-through? 

From  Figure  6.4-1,  we  determine  that  programming/ design  standards  are  established 
and  maintained  by  the  CD  and  the  TL.  Furthermore,  the  standards  are  used  by  the 
CD  and  AD  to  assess  status,  progress  and  performance. 

Structured  walk-through  procedure  is  established  and  controlled  by  the  TD,  AD  and 
TL.  The  results  of  the  walk-throughs  are  reviewed  and  interpreted  by  the  TD  and 
AD. 


Incorporating  these  elements  into  a  generalized  model,  Figure  7.1-3  results. 
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Figure  7.1-3  GENERALIZED  MODEL  OF  SCENARIO  B 


This  model  can  be  used  to  illustrate  the  development  process  in  the  event  of  a 
software  problem  or  question.  For  example,  suppose  the  Chief  Designer  (CD),  while 
reviewing  a  software  module,  detected  the  use  of  a  non-standard  programming 
practice.  He  knows  that  the  use  of  non-standard  programming  practices  often 
effects  the  portability  of  software.  In  essence,  the  software  could  become  machine 
dependent.  The  objective  of  machine  independence  is  supported  by  the  use  of 
programming  standards  and  walk-throughs.  The  CD  and  AD  are  notified  of  the 
standards  violation  and  are  held  responsible  for  resolving  the  problem.  They  would 
check  the  standards  manuals,  reports  from  any  previous  auditing,  and  reports 
generated  from  the  associated  walk-through  review.  The  walk-through  review 
report  might  contain  an  indication  of  why  the  non-standard  practice  was  used.  The 
CD  and  AD  must  determine  whether  or  not  a  valid  reason  exists  to  allow  usage  of 
non-standard  practices.  Depending  on  their  determination,  the  practice  will  either 
be  changed  or  made  highly  visible  through  documentation. 
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7.2  SUMMARY 


The  generalized  management  tool  model  developed  in  7.1  can  be  used  to  summarize 
the  studied  relationships  between  SPP  Indicators,  program  objectives,  management 
techniques,  MT  Indicators,  and  key  responsibilities.  This  model  illustrates  the 
studied  relationships  but  does  not  incorporate  all  the  elements  of  planning,  produc¬ 
tion  and  assessment  of  the  software  development  process.  These  elements  will  be 
the  topic  of  future  research. 
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APPENDIX  A 

SUMMARY  OF  RESPONSES  TO  PART  ONE  OF  SURVEY 


DEVELOPMENT/ 

MANAGEMENT 

OBJECTIVES 

NUMBER  OF  RESPONSES 

HIGH 

MEDIUM 

LOW 

UNCERTAIN 

ON  SCHEDULE 

5 

5 

3 

UNDER  COST 

4 

4 

5 

STATUS  VISIBILITY 

4 

2 

6 

1 

PROBLEM  RECOGNITION 

AND  CORRECTION 

4 

7 

2 

CONTRACTOR  COMMITMENT 

6 

5 

2 

USER  INVOLVEMENT 

6 

6 

1 

USEFULNESS  DEMONSTRATION 

7 

4 

2 

SATISFY  DIVERSE  NEEDS 

9 

- 

1 

STANDARD/PROCEDURE 

COMPLIANCE 

2 

3 

7 

1 

RELIABILITY  AND 

DEPENDABILITY 

7 

2 

3 

1 

CONFIGURATION  MANAGEMENT 

4 

4 

5 

MACHINE  INDEPENDENCE 

2 

2 

9 

MAINTAINABILITY 

4 

3 

5 

1 
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Appendix  F 


Hypothetical  IPAD  Situation  Scenarios 


A  Critical  Design/Development  Review  (CDR)  was  scheduled  to  review 
detailed  design  documentation.  Present  at  the  review  were  members  of 
the  IPAD  technical  and  management  teams  and  representatives  from 
NASA.  During  the  CDR,  NASA  proposed  a  change  to  the  IPAD 
requirements.  This  change  was  requested  in  order  to  restate  a  require¬ 
ment  to  meet  a  need  which  had  been  thought  to  be  covered  by  a 
combination  of  other  requirements.  IPAD  team  members  agreed  that 
the  need  would  not  be  satisfied  by  the  current  requirements.  The 
resulting  action  item  was  to  evaluate,  formally  state  and  incorporate  the 
proposed  requirements  change. 

A  meeting  of  the  IPAD  Technical  Advisory  Board  (ITAB)  was  scheduled 
and  held  to  comment  on  and  review  the  Reference  Design  Process 
Document.  During  this  meeting,  one  of  the  ITAB  members  pointed  out  a 
deficiency  in  the  design  process.  The  committee  agreed  that  the 
deficiency  should  be  reported  to  IPAD  and  a  letter  to  the  IPAD  Program 
Manager  and  the  NASA  Project  Manager  was  written. 

A  Preliminary  Design  Review  (PDR)  was  scheduled  and  held.  Members 
of  IPAD  Management  and  technical  teams  and  NASA  were  present. 
Some  members  of  ITAB  were  also  involved.  The  customer  (NASA)  was 
concerned  about  what  had  been  "designed  into"  IPAD  software  to  insure 
reliability  and  maintainability.  Members  of  IPAD  assured  NASA  that  the 
use  of  rigid  programming  and  engineering  standards  would  make  IPAD 
easily  maintainable,  and  that  other  techniques  were  being  used  to  ensure 
reliability.  NASA  indicated  a  strong  interest  in  development  and 


application  of  thorough  testing  procedures  and  requested  a  demon¬ 
stration  that  sufficient  test  case  coverage  was  achieved.  It  was 
determined  that  the  PDR  action  item  was  a)  to  define  an  acceptable 
measure  of  testing  sufficiency  and  b)  to  update  IPAD  management  and 
technical  plans  accordingly. 
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Appendix  G 


Process  of  the  Action  Item  (AI)  for  Scenario  // 1 
Step(s)  Action  Taken 

1.  NASA  CDR  Board  Chairman  gives  Action 
Item  to  IPAD  Program  Manager. 

2.  IPAD  Program  Manager  turns  AI  over  to 
Assistant  Program  Manager  for  technical 
evaluation  and  routing. 

3.  Assistant  Program  Manager  forwards  AI  to 
Software  Development  Manager  for  evalu¬ 
ation  and  task  delegation. 

4.  Software  Development  Manager  assigns  task 
of  Formal  Statement  to  manager  responsible 
for  IPAD  Requirements  and  Requirements 
Documentation. 

5.  Requirements  Manager  delegates  Formal 
Statement  responsibility  to  IPAD  computing 
staff  member. 

6.  IPAD  computing  staff  member  prepares 
Formal  Statement  of  Requirement  for  Re¬ 
quirements  Manager,  and  contacts  Program 
Control  Coordinator  (PCC)  for  CCR  Form 
because  Requirements  Document  is  under 
Configuration  Control. 
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Appendix  G  (Continued) 

Step(s)  Action  Taken 

7.  The  PCC  and  computing  staff  member  fill 
out  the  Configuration  Change  Request  (CCR) 
and  Impact  Summary  and  submit  them  to  the 
Requirements  Manager  for  his  appraisal  of 
initial  impact. 

8.  Requirements  Manager  fills  out  anticipated 
impact  on  the  CCR  Form  and  Management 
Summary  on  Impact  Analysis  Form  and  re¬ 
turns  the  CCR  package  to  the  PCC. 

9.  The  PCC  logs  the  CCR  in  Configuration 
Control  Log.  The  PCC  makes  10  copies  of 
proposed  Change  Package  and  distributes  this 
to  managers  with  notification  of  Configur¬ 
ation  Change  Board  Meeting. 

10.  The  Configuration  Change  Board  (CCB)  con¬ 
siders  the  proposed  Change  Package  and 
orders  a  full  impact  analysis  if  they  think  it 
necessary,  or  they  pass  and  sign  the  CCR  and 
Impact  Summary  forms  and  the  package  is 
routed  on  by  the  PCC. 

11.  The  PCC  logs  the  Board  decision  and  if 
passed  he  will  route  to  the  Program  Manager 
for  concurrence  signature. 
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Appendix  G  (Continued) 

Step(s)  Action  Taken 

12.  Program  Manager  reviews  the  CCR  Package, 
signs  and  returns  to  the  PCC  for  forwarding 
to  NASA.  (Changes  to  Formal  Configured 
Items  are  Level  I  and  require  NASA  approv¬ 
al.) 

13.  The  PCC  takes  the  approved  package,  logs 
the  Program  Manager's  signature  and  date, 
makes  a  copy  of  the  Package  in  its  approved 
form  and  sends  the  original  to  NASA. 

14.  NASA  considers  change  package  and  either 
rejects  or  accepts  it.  If  accepted  it  will  be 
returned  to  the  PCC  at  Boeing,  usually  within 
30  days. 

15.  The  PCC  then  takes  the  Accepted  Package 
and  turns  it  over  to  an  ATMS  (Automated 
Text  Management  System)  typist  and  she 
makes  the  changes  or  in  this  case  additions  to 
the  Master  File  for  this  document  on  ATMS, 
returning  the  text  and  the  CCR  to  the  PCC 
for  collation  with  revised  graphics  originals 
and  the  PCC  will  route  revised  doucment 
master  to  responsible  manager  for  his  ap¬ 
proval.  If  the  responsible  manager  is  satis¬ 
fied  with  the  document  he  will  O.K.  a  release 
of  the  revised  document. 
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Appendix  G  (Continued)  j 

Step(s)  Action  Taken 

16.  The  PCC  will  then  send  the  revised  document  j 

master  to  printing  where  200  (approximately) 
copies  will  be  made  and  sent  back  to  the 
PCC.  It  is  then  his  responsibility  to  see  that 
everyone  who  has  been  issued  the  document 
in  the  past  receives  an  updated  copy. 


Appendix  G  (Continued) 


Actions  in  Response  to  Scenario  #2 
>(s)  Action  Taken 


ITAB  Interface  Manager  sends  action  item  to 
Program  Manager  and  NASA. 

Program  Manager  assigns  the  Engineering 
Development  Manager  to  correct  Reference 
Design  Process  document  as  required  and 
establish  modifications  (if  any)  required  for 
the  requirements  documentation. 

Engineering  Development  Manager  opens  an 
authorized  study  account  number  and  assigns 
study  and  plan  task  to  engineer. 

PCC  and  Engineering  Development  Manager 
fill  out  CCR  and  cost  and  impact  estimates 
are  made  (see  response  to  Scenario  it  l,  steps 
6-9). 


CCB  meets  and  decides  what  to  do.  Assume 
document  update  and  update  of  dependent 
documents  is  proposed. 

Engineering  Development  Manager  opens  an 
authorized  account  number  for  a  doing  task, 
assigns  task  to  engineer.  Result  is  a  CCR  of 
document  updates. 


Appendix  G  (Continued) 


Step(s) 

Action  Taken 

7. 

CCB  meets  and  approves  the  update. 

8. 

Program  Manager  and  NASA  approve. 

9. 

Configuration  control  function  carries  out 
document  update. 

10. 

Changes  reported  to  next  ITAB  meeting. 
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Appendix  G  (continued) 


Actions  in  Response  to  Scenario  #3 


Functional  events  or  steps  to  resolve  the  third  scenario  were  not  detailed.  In 
essence,  very  little  could  be  learned  from  the  third  scenario  which  had  not  already 
been  illustrated  by  Scenarios  #1  and  #2. 

A  group  would  have  been  assembled  to  study  the  existing  testing  procedures.  The 
Development  Test  Plan  and  the  Acceptance  Test  Plan  would  have  been  reviewed.  If 
the  existing  procedures  had  been  satisfactory  the  procedures  would  have  been 
documented  and  resubmitted  to  the  customer.  If  unsatisfactory,  new  procedures 
would  have  been  studied,  developed,  approved  and  incorporated  into  existing  Test 
Plans.  The  procedure  for  change  procedure  has  already  been  documented  as  a  part 
of  Scenario  // 1. 
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